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Executive Summary 


The Technology Immersion Pilot (TIP) sets forth a vision for technology immersion in Texas public 
schools. The Texas Education Agency (TEA) directed nearly $14 million in federal Title II, Part D 
monies toward funding a wireless learning environment for high-need middle schools through a 
competitive grant process. A concurrent research project funded hy a federal Evaluating State 
Educational Technology Programs grant is evaluating whether student achievement improves over 
time as a result of exposure to technology immersion. The Texas Center for Educational Research 
(TCER) — a non-profit research organization in Austin — is the TEA’s primary partner in this landmark 
effort. 

The overarching purpose of the study is to conduct a scientifically based evaluation at the state level to 
test the effectiveness of technology immersion in increasing middle school students’ achievement in 
core academic subjects. Technology immersion encompasses multiple components, including a laptop 
computer for every middle school student and teacher, wireless access throughout the campus, online 
curricular and assessment resources, professional development and ongoing pedagogical support for 
curricular integration of technology resources, and technical support to maintain an immersed campus. 

Technology Immersion 

As a way to ensure consistent interpretation of technology immersion and comparability across sites, 
the TEA issued a Request for Qualifications (REQ) that allowed commercial vendors to apply to 
become providers of technology immersion packages. Successful vendor applicants to the REQ had to 
include the following six components in their plan: 

• A wireless mobile computing device for each educator and student on an immersed campus 
to ensure on-demand access to technology; 

• Productivity, communication, and presentation software for use as a learning tool; 

• Online instructional resources that support the state curriculum in English language arts, 
mathematics, science, and social studies; 

• Online assessment tools to diagnose students’ strengths and weaknesses or to assess their 
progress in mastery of the core curriculum; 

• Professional development for teachers to help them integrate technology into teaching, 
learning, and the curriculum; and 

• Initial and ongoing technical support for all parts of the package. 

Through a competitive application submission and expert-review process, the TEA selected three lead 
vendors as providers of technology immersion packages (Dell Computer Inc., Apple Computer Inc., 
and Region 1 Education Service Center [ESC]). Prices for packages varied according to the numbers 
of students and teachers, the type of laptop computer, and the vendor provider. Package costs ranged 
from about $1,100 to $1,600 per student. Of the 22 immersion sites, 6 middle schools selected the 
Apple package, 15 selected the Dell package, and 1 school selected the Region 1 ESC package (Dell 
computer). 
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Methodology 


Evaluation Design 

The evaluation employs a quasi-experimental research design with 44 middle schools assigned to 
either treatment or control groups (22 schools in each). Researchers will examine the relationships that 
exist among contextual conditions, technology immersion, intervening factors (school, teacher, and 
student), and student achievement. The research also will determine the impact of immersion on 
student achievement in core subject areas as measured hy the Texas Assessment of Knowledge and 
Skills (TAKS). We investigated six research questions in the first year: 

• What are the baseline characteristics of participating schools? 

• How is technology immersion implemented? 

• What is the effect of technology immersion on schools? 

• What is the effect of technology immersion on teachers and teaching? 

• What is the effect of technology immersion on students and learning? and 

• Does technology immersion impact student achievement? 

The Theoretical Framework for Technology Immersion guides the evaluation. The experimental 
research design allows an estimate of the effects of the intervention, which is the difference between 
the treatment and control groups. The framework postulates a linear sequence of causal relationships. 
First, experimental schools are to be “immersed” in technology through the introduction of technology 
immersion components. Given quality implementation, school-level improvements are expected for 
measures of classroom technology integration, technical support, innovative culture, and parent and 
community support. Leadership and system support drives progress toward full immersion. 

An improved school environment for technology should then lead to teachers who have greater 
technology proficiency, use technology more often for their own professional productivity, collaborate 
more with their peers, have students use technology more and in new ways in their classrooms, and 
use laptops and digital resources to increase the intellectual challenge of lessons. In turn, these 
improved school and classroom conditions should lead students to greater technology proficiency, 
more opportunities for peer collaboration, greater personal self-direction, and stronger engagement in 
school and learning. Student mediating variables presumably contribute to increased academic 
performance as measured by standardized test scores. In the framework, links are also shown between 
student achievement and student, family, and school characteristics, which exert their own influence 
on learning. 

Participating Sites 

Interested districts and associated middle schools responded to a Request for Application (RFA) 
offered by the TEA in spring 2004 to become technology immersion schools. Applicants to become 
TIP sites had to meet eligibility requirements for Title II, Part D funds (i.e., high-need due to children 
from families with incomes below the poverty line, schools identified for improvement, or schools 
with substantial need for technology). Twenty-two technology immersion schools, selected through 
the competitive grant process, were matched by researchers with 22 control schools on key 
characteristics, including size, regional location, demographics, and student achievement. 

The TIP grants targeted high-need schools, thus nearly 70% of students in the study come from 
economically disadvantaged backgrounds, with many schools in rural or isolated locations. Students 
are ethnically diverse, roughly 56% Hispanic and 9% African American. TIP Middle schools are 


ii 



highly concentrated in rural and very small districts across the state. Still, about a third of the districts 
and schools are in large cities or suburban locations in or around cities. The sample also includes 
campus charter schools (one each for the treatment and control group) located in a major urban 
district. 

Three groups or cohorts of students will be followed in the study, with Cohort 1 followed for four 
years. Cohort 2 for three years, and Cohort 3 for two years. In 2004-05, data collection activities 
centered on the initial sixth-grade cohort, which included 5,564 students (2,570 at immersed and 2,994 
at control campuses). About 1,304 teachers participated in the study (622 at immersed and 682 at 
control campuses). 

Data Collection and Analysis 

Data collection involved a mix of qualitative and quantitative data sources. Researchers conducted site 
visits in each of the middle schools in fall 2004 and spring 2005. For this report, we concentrate on 
site- visit data gathered through observations in a sample of sixth-grade classrooms (English/language 
arts, mathematics, social studies, and science). Additional measures, administered as pre- and post- 
measures in fall and spring, include a Campus Technology Inventory completed by the campus 
technology coordinator, teacher online surveys, and student paper-and-pencil surveys. Additionally, 
we gathered school and student demographic, attendance, and achievement data from the Texas Public 
Information Management System (PEIMS) and Academic Excellence Indicator System (AEIS). In 
spring 2005, individual middle schools submitted student-level data on disciplinary actions. 

We analyzed the effects of immersion on teachers’ and students’ self-reported perceptions of 
technology and proficiencies and students’ TAKS achievement using two-level hierarchical linear 
models (HEM). Eor various analyses contrasting teachers or students in immersed and control schools 
after one school year of implementation, we used important teacher characteristics (fall survey scale 
scores, experience, technology certification, gender) and student characteristics (fall survey scale 
scores, prior achievement, economic and minority characteristics, and gender) as control variables. We 
also calculated effect sizes in standard deviation units (usually Cohen’s d). The interpretation is that an 
effect greater than 0.5 is large, 0.5 to 0.3 is moderate, 0.3-0. 1 is small, and less than 0.1 is trivial. 

Major Findings 

Eirst-year results reveal positive effects of technology immersion on schools (leadership and system 
support, innovative culture, classroom integration, parent and community support), teachers 
(proficiency and productivity, technology use and integration, collaboration), and students (technology 
proficiency and use, small-group work, school satisfaction, and behavior). In most cases, the sizes of 
effects suggest that the impacts of technology immersion are of both statistical and practical 
importance. In contrast to positive effects on school, teacher, and student mediating variables, there 
were no statistically significant effects of immersion in the first year on either reading or mathematics 
achievement for sixth graders, who are members of a student cohort that will be followed through 
eighth grade. Overall, positive findings are compelling in light of evidence indicating that the level of 
implementation in the first year for 20 of the 22 middle schools was only partial immersion rather than 
substantial (2 schools) ox full immersion (no schools). Additional details for key findings are provided 
below. 

First-Year Implementation 

Researchers used rating scales to identify four levels of immersion: minimal {\), partial (2), 
substantial (3), and full (4). The overall level of Technology Immersion was a composite score derived 
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from values for four domains: (a) Robust Access to Technology, (b) Technical and Pedagogical 
Support, (c) Professional Development, and (d) Resource Utilization and related indicators. Scores 
came from various data sources including vendor records, interviews, focus groups, surveys, and grant 
documents. 

In the first year, almost all middle schools achieved only partial immersion. Middle schools 
struggled in the initial year to accommodate the complex demands of technology immersion within the 
existing school environment. As might be expected, no campus reached full immersion. The two 
middle schools that made greater strides toward immersion than others {substantial immersion) had 
stronger district and campus leadership and invested more time and resources in professional 
development. 

In general, first-year implementation was affected by a number of school and contextual factors. First, 
time for planning was insufficient due to grant-related logistical procedures. Furthermore, many 
middle schools, which were housed in older buildings, encountered problems with outdated 
infrastructures and technical problems with wireless networks and Internet connectivity. Districts and 
campuses also had to grapple with myriad policies and practices related to laptop access and use. The 
greatest barriers to implementation, however, involved people. Teachers were at different stages of 
readiness for immersion and their receptivity varied. Varying abilities and attitudes, coupled with 
teachers’ perceived pressures to improve students’ scores on the TAKS, made many teachers reluctant 
to try new and untested instructional methods and materials in the first year. Additionally, leadership 
at both the district and campus levels emerged as a critical factor driving or limiting progress. 

Effects of Immersion on Schools 

Technology immersion positively affects the school culture, including factors such as innovation, 
collaboration, leadership, parent and community support, and students’ school satisfaction. 

Technology immersion had a statistically significant effect on teachers’ perceptions of four school- 
level factors. Since immersed schools received a wealth of technology resources, it was predictable to 
find that immersed teachers perceived greater availability and use of resources for Classroom 
Technology Integration than control teachers (effect size of 0.56). Teachers in immersed schools also 
reported stronger Leadership and System Support for technology (effect size of 0.20). More 
remarkable, however, was immersed teachers’ perceptions of a more Innovative Culture in their 
middle schools (effect size of 0.35). In particular, teachers at immersed schools were more likely than 
control teachers to share an understanding about the use of technology to enhance student learning, 
and they were less afraid to learn about and try new technologies in their classes. The infusion of 
technology also increased Collaboration among treatment teachers (effect size of 0.41). Teacher 
interactions at immersed schools significantly more often than at control supported improvements in 
instructional practices and exchanges of information about students and their learning. 

The implementation of technology immersion also generated a great deal of excitement in schools and 
communities. This likely contributed to immersed teachers’ belief that their schools have stronger 
Parent and Community Support for technology (effect size of 0.49). Sixth-grade students at immersed 
middle schools also expressed significantly higher levels of School Satisfaction than control students 
(effect size of 0.13). Treatment students were more likely to be satisfied with their school work, 
consider learning more important than the grade received, and see a connection between school work 
and their future life and work. 
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Effects of Immersion on Teachers 


Teachers at immersed schools perceive themselves as more technology proficient than control 
teachers and use technology more productively to support professional practices. In a self- 
assessment of Technology Proficiency in spring, teachers at immersed schools considered themselves 
to he significantly more technology literate than control teachers (effect size of 0.16). Although 
teachers were equally likely to he proficient in technology operations, teachers at immersed schools 
reported greater pedagogical skills in areas such as creating electronic presentations, teaching 
copyright issues, creating technology-integrated lesson plans, and using technology for collegial 
collahoration. Immersed teachers also began to use technology significantly more often than control 
teachers for administrative and classroom management purposes. Treatment teachers reported greater 
use of technology for Professional Productivity (effect size of 0.37) on indicators such as 
communicating with students, posting information on a wehsite, administering an online assessment, 
and accessing model lesson plans integrating technology. 

Teachers at immersed schools have students use technology more often and they report the use 
of more innovative and learner-centered practices compared to control teachers. With increased 
access to technology, teachers at immersed schools compared to control reported in spring that their 
Students Use Technology significantly more often in their classrooms (effect size of 0.70). For 
example, students more often express themselves in writing (using a word processor), learn and 
practice skills, and conduct Internet research on an assigned topic. Still, treatment teachers’ responses 
suggest that students may do such activities infrequently (i.e., only once or twice a month). Teachers 
at immersed schools also expressed stronger support for Technology Integration (effect size of 0.73). 
For example, they were more likely than control teachers to report that they allocate time for students 
to practice computer skills, plan computer-related activities to improve students’ basic skills, use 
cutting-edge technology, and use computers to promote students’ problem solving and critical 
thinking. Immersed teachers also expressed a stronger affiliation with Learner-Centered Instruction 
(effect size of 0.30). Immersed teachers, for instance, were more likely than control to indicate that 
students establish individual learning goals, engage in experiential learning, and have real-world 
experiences. 

Although teachers at immersed schools use technology more, their lessons typically lack 
intellectual challenge. Technology immersion’s theorized impact on student achievement hinges not 
just on more frequent technology use, but also on technology’s facilitation of more rigorous and 
authentic learning (e.g., high-level thinking, concept formation, inquiry and investigation, access to 
and use of information, exposure to places/resources beyond the classroom, and real-world learning). 
Thus, during fall and spring observations in sixth-grade classrooms, researchers rated the Intellectual 
Challenge of lessons. Rating scales (developed by Newmann, Secada, and Wehlage, 1995) gauged 
Higher Order Thinking, Disciplined Inquiry (Deep Knowledge and Substantive Conversation), and 
Value Beyond School. 

Pre- and post-results for 58 immersed and 57 control teachers revealed no statistically significant 
differences between comparison groups in spring 2005. Nevertheless, fall-to- spring comparisons 
revealed that teachers in immersed classrooms provided slightly more challenging lessons in spring, 
whereas control teachers taught less challenging lessons. More noteworthy, however, was the low 
level of intellectual challenge in class activities for both comparison groups (about 1.6 on the 5-point 
intellectual challenge scale). In many of the observed sixth-grade classrooms, with or without laptop 
use, teachers concentrated on lower order factual knowledge and skills. Lessons frequently involved 
multiple-choice or short-answer worksheets focused on the acquisition of basic skills rather than more 
complex endeavors and higher order thinking. Additionally, lessons often featured brief instructional 
segments across a variety of learning objectives rather than in-depth focus on a topic or concept. 



Moreover, teachers rarely helped students to understand the relevance of their learning or made 
connections with students’ prior experiences. Findings from classroom observations are important 
because of the established link between more challenging and authentic pedagogy and academic 
achievement (Newman & Associates, 1996; Newmann, Bryk, & Nagoaka, 2001). If abundant access 
to technology fails to elevate the quality of students’ learning experiences, the likelihood of a positive 
impact on student achievement may be diminished. 

A major challenge for teachers in the first year was simultaneously learning how to use 
technology and finding time to integrate laptops and digital resources into existing practices. 

Although teachers at immersed schools, as a whole, made substantial progress in the first year, teacher 
proficiency and laptop use varied greatly by teacher, subject area, and school. Decisions about how 
and how often laptops were used for teaching and learning depended on each teacher’s readiness and 
preference. Survey results show that more experienced teachers and male teachers in middle schools 
viewed themselves as less proficient, used technology significantly less often, and expressed lower 
level of support for technology integration. 

Information from classroom observations and field work also suggest that in the initial stages of 
implementation, most teachers maintained their existing pedagogical practices. Teachers typically had 
students use laptops to do the same kinds of activities they previously had completed with paper and 
pencil, such as completing worksheets, typing vocabulary words and definitions, or reviewing for 
multiple-choice tests. This finding is consistent with research showing that teachers progress through 
developmental stages while learning to create technology-infused classroom environments. Many 
teachers at immersed campuses appeared to be at the adoption or adaptation phases, as they were 
using technology to support traditional instruction or integrating new technology into traditional 
classroom practice (Apple Computer Inc., 1995). 

Effects of Immersion on Students 

Students at immersed campuses are more highly engaged in school than control students. 

Increased student engagement is one of the most frequently cited benefits in the research literature for 
one-to-one computing. Likewise, during campus visits, administrators, teachers, and students at 
immersed campuses cited greater student interest and motivation for school and learning as positive 
effects. Other findings corroborate anecdotal perceptions. Surveyed sixth-graders at immersed 
campuses in spring expressed significantly higher levels of satisfaction with their middle schools than 
control students. Additionally, sixth graders at immersed schools were sent to the office for 
disciplinary reasons at a significantly lower rate and had fewer school suspensions than students at 
control schools. Effect sizes for school satisfaction (0.13) and disciplinary measures (0.16 and 0.06), 
however, were small. Also, for another indicator of engagement, the school attendance rate, there was 
no apparent boost for immersed students (effect size of 0.08). 

Technology immersion positively affects sixth graders’ technology proficiency and opportunity 
to use technology. As anticipated, sixth-grade students at immersed middle schools rated their 
Technology Proficiency significantly higher than control students (effect size of 0.47) on items 
measuring the Texas Technology Applications standards. Immersed students felt more capable of 
performing tasks such as sending an email attachment, creating a presentation, managing documents, 
using spreadsheets for graphs, and keeping track of websites. Immersed students’ increased 
proficiency apparently stems from more frequent technology use. Similar to their teachers, surveyed 
sixth graders at immersed schools reported significantly more frequent Technology Use in Core 
Subjects than control students (effect size of 0.96). Flowever, despite large and important increases, 
immersed students’ technology use varied across classrooms and content areas. Treatment students 
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reported using technology most often in reading/English language arts, science, and social studies 
classes (nearly once or twice a week) and least often in math classes (about once or twice a month). 


There was no apparent effect of technology immersion on student self-direction. We theorized 
that sixth graders’ opportunities for independent and self-guided learning afforded through one-to-one 
technology would positively affect students’ personal self-direction. Students completed the Style of 
Learning Inventory as a measure of self-directed learning, including processes such as forethought, 
performance/volition control, and self-reflection. Findings in spring showed there was no significant 
difference between the Self-Directed Learning scale scores for sixth graders in immersed and control 
schools (effect size of 0.06). Nevertheless, changes in students’ perceptions of their self-direction may 
emerge as they progress to higher grade levels and perhaps use their laptops in more and better ways. 

Effects of Immersion on Academic Achievement 

There was no significant effect of technology immersion on sixth graders achievement in reading 
or mathematics. The ultimate goal of technology immersion is increasing middle school students’ 
achievement in core academic subjects as measured by the state assessment (TAKS). In Texas, sixth 
graders complete TAKS assessments for reading and mathematics. We found that after one academic 
year of implementation, there were no positive effects of immersion on either reading or mathematics 
scores. After controlling for prior achievement and other important student characteristics, there were 
no significant differences in the spring 2005 reading or mathematics TAKS z scores of students in 
immersed and control schools. In fact, students in immersed schools had slightly lower scores than 
comparison students. 

Several factors help to explain the discontinuity between the many positive effects noted for schools, 
teachers, and students at immersed campuses and the absence of a positive effect on student 
achievement outcomes. First, implementation fidelity was an important factor. Limited project 
implementation almost certainly influenced outcomes (e.g., the small number of days that students 
actually had laptops, the minimal use of digital resources). In our theoretical model, we hypothesized 
that students in fully immersed schools would experience school and classroom environments that 
would lead to changes in students, which in turn, would lead to increased achievement. While we 
found noteworthy improvements in some areas (e.g., changes in teacher proficiency and technology 
use, improvements in students’ proficiency and school engagement), there were no positive effects on 
students’ personal self-directed learning, and based on classroom observations, the availability of 
laptops did not lead to significantly greater opportunities for students to experience intellectually 
challenging lessons or to do more challenging school work. 

Furthermore, although technology use increased in the first year and surpassed control schools, 
laptops were used infrequently for learning in core subject classes, especially mathematics. Using 
laptops for lessons once or twice a week, or once or twice a month in math classes, may be insufficient 
to make a difference in achievement. Unfortunately, students in Texas middle schools do not complete 
social studies assessment until eighth grade or a science assessment until tenth grade, so we did not 
have academic outcome measures for those content areas. 

It is also important to remember that this is a longitudinal study, and while we expected that some 
impacts might emerge in the first year, it was also considered likely that changes in student academic 
performance would require more than one year to surface. Additionally, the findings reported here 
represent only a first step in analyzing first-year data. Additional analyses will further examine the 
relationships among school, teacher, and student mediating variables and academic achievement. We 
also intend to delve more deeply into the relationships among the fidelity of implementation, 
mediating variables, and outcomes. 
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1. Introduction 


The Technology Immersion Pilot (TIP) sets forth a vision for technology immersion in Texas public 
schools. The Texas Education Agency (TEA) directed nearly $14 million in federal Title II, Part D 
monies toward funding a wireless learning environment for high-need middle schools through a 
competitive grant process. A concurrent research project funded hy a federal Evaluating State 
Educational Technology Programs grant is evaluating whether student achievement improves over 
time as a result of exposure to technology immersion. The Texas Center for Educational Research 
(TCER) — a non-profit research organization in Austin — is the TEA’s primary partner in this landmark 
effort. 

The overarching purpose of the study is to conduct a scientifically based evaluation at the state level to 
test the effectiveness of technology immersion in increasing middle school students’ achievement in 
core academic subjects. Technology immersion encompasses multiple components, including a laptop 
computer for every middle school student and teacher, wireless access throughout the campus, online 
curricular and assessment resources, professional development and ongoing pedagogical support for 
curricular integration of technology resources, and technical support to maintain an immersed campus. 
The evaluation, with 22 experimental and 22 control sites, will examine the relationships that exist 
among contextual conditions, technology immersion, intervening factors (school, teacher, and 
student), and student achievement. Moreover, the research will determine the impact of immersion on 
student achievement in core subject areas as measured by the Texas Assessment of Knowledge and 
Skills (TAKS). 


Background 

The conception of educational technology held by many educators, leaders, and policymakers has 
shifted in recent years from the use of particular technology applications to technology’s incorporation 
into every aspect of the educational environment. Changing views reflect our growing understanding 
of how students learn and how to create technology-infused environments that enhance teaching and 
learning. Cognitive science and other research reveal that children learn more when they are engaged 
in meaningful, relevant, and intellectually stimulating work. Moreover, learner- and knowledge- 
centered environments can help students make connections between their previous knowledge and 
current academic tasks, allowing students to grasp more complex concepts (Bransford, Brown, & 
Cocking, 2003; Newmann, Bryk, & Nagoaka, 2001). In the present view of learning, “the use of 
technology is not an add-on, but an integral part of the students’ quest for knowledge and a tool 
through which students research, organize, and share information” (Johnston & Cooley, 2001, p. 25). 

Many also believe that when applied to well-defined educational objectives and integrated into the 
curriculum by trained teachers, educational technology can help students develop the competencies 
needed for the 21st century. The CEO Eorum concluded that children who are growing up in the 
Digital Age must have different competencies, and technology serves as a bridge to academic 
achievement through its support for the acquisition of skills, including digital age literacy, inventive 
thinking, effective communication, and high productivity (CEO Eorum, 2001; Eempke, Couglin, 
Thandani, & Martin, 2003). 

Similarly, Texas has long recognized that the state’s success is tied to the provision of social, 
intellectual, and economic opportunities of the Digital Age. Preparing for the 21st century means that 
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Texas students must learn different ways to work with tools, information, and people. The Texas 
Long-Range Plan for Technology, 1996-2010, outlines a comprehensive approach for the effective 
integration of technology within schools across four major domains: teaching and learning, educator 
preparation and development, administration and support services, and infrastructure for technology 
(Texas Education Agency [TEA], 2002). This systemic approach has been further refined through 
Texas Senate Bill 396 as “technology immersion.” Technology immersion calls for the provision of a 
wireless mobile computing device for each student in a school, the use of technology-based learning 
resources, training teachers to integrate technology into the classroom, and the provision of support for 
effective technology use. 


Theory of Technology Immersion 

In operationalizing technology immersion (i.e., specifying the critical components), the TEA 
considered existing research on educational technology as well as practical wisdom gained through 
numerous pilot studies and statewide technology initiatives. The technology immersion model 
assumes that effective technology use in schools and classrooms requires robust access to technology, 
technical and pedagogical support for implementation, professional development for educators in 
using technology effectively, and readily available curricular and assessment resources to support the 
state’s curriculum in the core subjects (English language arts, mathematics, science, and social 
studies). 

Robust Access to Technology 

One indicator of robust access to technology is the ratio of students to computers. Believing that 
increased access to computers in schools leads to increased technology use, considerable effort has 
gone into reducing student-to-computer ratios. Over time, the ratio has dropped nationally from 125:1 
in 1983 to 6:1 in 1998, and 3:1 in 2004 (Market Data Retrieval, 1999; Education Week, 2005). 
Combined with hardware availability, schools also have built their infrastructure for technology. In 
2004, 99% of public schools in the United States had access to the Internet; an increase from only 35% 
in 1994 (NCES, 2000a; Education Week, 2005). 

Despite increased access, research shows that students’ technology use is relatively low. While 
technology use in schools has increased, students use computers for only a small portion of each 
school day (Cuban, Kirkpatrick, & Peck, 2001; Russell, O’Brien, Bebell, & O’Dwyer, 2003). 
Moreover, when students use technology, it is most frequently in a computer lab setting or in 
technology courses; computer and Internet use are less common in core content areas, particularly 
mathematics (NCES, 2000b; Becker, 2001). Similar to national trends, students’ technology access has 
increased overall in Texas schools, but many computers are in computer labs, thus limiting regular 
student use. On a statewide survey, less than 10% of grades 6 to 12 students reported using technology 
most often in their classrooms. Eurthermore, teachers reported an average of only 2.9 classroom 
computers, insufficient to allow every student computer access. Accordingly, teachers and principals 
cite insufficient numbers of classroom computers as a barrier to instructional use (Shapley, Benner, 
Heikes, & Pieper, 2002). 

Differences in technology access also pose challenges for economically disadvantaged, minority, and 
low-achieving students. Eow-income students have fewer opportunities than their more advantaged 
peers to develop effective technology skills and use technology to enhance learning both at school and 
at home (Shields & Behrman, 2000). A Maryland study found that regular technology use for research 
decreased as school poverty level increased (Education Week, 2001). Minority students also are less 
often exposed to technology, primarily due to disparities in technology access, resulting in fewer 
opportunities for use (Reid, 2001; Shapley, et ah, 2002). 
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As a way to counteract prevailing conditions, technology immersion calls for one-to-one student 
access to technology. The Texas project is not unique in its quest for one-to-one computing. As 
computer technologies have become more affordable and accessible, large-scale projects have begun 
to appear with each student in a school, grade level, or classroom receiving his or her own computing 
device. Some states and school districts across the United States, including Maine, Michigan, New 
Hampshire, Vermont, Henrico County in Virginia, Beaufort County in South Carolina, and Cobb 
County in Georgia have implemented one-to-one computing projects (Zucker, 2004; SRI International, 
2005). Although the technology immersion pilot is similar to other laptop projects in its provision of 
one-to-one computing, it is unique in its focus on immersing entire schools in technology and 
simultaneously providing technical and pedagogical support, professional development, and digital 
resources. 

Technical and Pedagogical Support 

Technology immersion assumes that increased access to and use of technology in schools requires a 
healthy technical infrastructure and adequate technical and pedagogical support. Schools must have 
electronic networks that are robust enough to support wireless laptops and digital content. Campus- 
based technical support is also vital, as many studies emphasize the importance of on-site access to 
support personnel who are responsible for assisting teachers in learning to use technology, 
troubleshooting technical problems, and effectively integrating technology into lessons (e.g., CEO 
Forum, 2001; Ringstaff & Kelley, 2002; Shapley et al., 2002). One study found that teachers in 
schools without on-site technology coordinators were two to three times more likely to identify the 
lack of technical support as a barrier to instructional technology implementation (NCES, 2000b). 

Other studies have found a strong relationship between the provision of quality technology support 
and teachers’ technology use and their changes in use over time (Ronnkvist, Dexter, & Anderson, 
2000), as well as their use of the Internet for instruction (NCES, 2002a). 

Disparities in access to technology support also emerge. Teachers at schools in low-socioeconomic 
neighborhoods report less technical and instructional support and less access to quality technology 
support services, such as one-on-one assistance or widespread peer support (Ronnkvist et al., 2000). 
District and school size in Texas is also associated with technology support. As district and campus 
enrollment increases, teachers more often report receiving technology-related support from a district 
instructional specialist and on-site support from technology coordinators and expert teachers. 
Conversely, as enrollment decreases, teachers more often receive support from a district rather than a 
campus technology coordinator (Shapley et al., 2002). 

In addition to technical assistance, ongoing pedagogical support for teachers’ efforts to use technology 
is crucial. A primary reason cited by teachers who do not use technology in their classrooms is a lack 
of experience with the technology (Rosen & Weil, 1995; Wenglinsky, 1998). According to White, 
Ringstaff, and Kelley, “As teachers begin using technology for more sophisticated purposes, 
instructional support is as essential as technical support” (2002, p. 9). Although professional 
development for teachers provides a start, ongoing, campus-based mentoring and coaching is also 
necessary. 

In light of the documented importance of support services, technology immersion requires the 
provision of ongoing technical and pedagogical support for schools and teachers. 

Professional Development 

Technology immersion also assumes that technology’s potential impact on student learning depends 
on teachers’ opportunities for effective professional development. A lack of professional development. 
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in fact, is cited as the most common barrier to effective technology integration (Charp, 1997; Office of 
Technology Assessment, 1995). Furthermore, research shows that effective professional development 
must be ongoing, relevant, and support teachers’ basic technology skill acquisition as well as 
understanding of effective integration methods (CEO Forum, 2000, 2001; Denton, Davis, & Strader, 
2001; Web-Based Education Commission, 2000). In addition, training should be of an adequate length 
to comprehensively investigate the training topics, provide time for practice and experimentation and 
be immediately available as needed (ACE, 1999; NCES, 1999; NCES, 2000b). Moreover, the constant 
evolution of instructional technology coupled with teacher preparation program limitations 
necessitates continuous technology-related professional development targeting both novice and 
experienced teachers. 

Professional development, as noted previously, should also include follow-up to support teachers as 
they implement new skills in the instructional setting (Apple Computer, Inc., 1995). Research shows 
that when a particular technology use is mastered by teachers over time or promoted through sustained 
professional development, it is more likely to survive (Zhao & Frank, 2003). Ongoing professional 
development is necessary not only to help teachers learn how to use new technology but also to learn 
how to provide meaningful technology-based instruction and activities in the classroom (Sulla, 1999). 

Teacher involvement in technology-related professional development has also been associated with 
positive outcomes. Teachers who participate in professional development more often use technology 
for instructional purposes and are more likely to identify technology as an essential resource (Becker, 
1999; NCES, 2002a). Moreover, as training participation increases, teacher reports of feeling well 
prepared to use technology for instruction increase as well (NCES, 2000b). Wenglinsky (1998) found 
that teachers who had received professional development with computers during the previous five 
years were more likely to use computers in effective ways than those who had not participated in such 
training. In view of overall evidence, professional development plays a critically important role in 
technology immersion. 

Curricular and Assessment Resources 

Technology’s impact on student academic achievement in an immersed school hinges on the 
availability of instructional and learning resources that support the state’s curriculum standards. These 
resources include productivity, communication, and presentation software that allow students and 
educators to use wireless laptops as a tool for teaching, learning, communication, and productivity. 
Additionally, digital resources (e.g., online, CD-ROMS, stored on local networks) provide a means to 
support more engaged, thoughtful, relevant, and personalized learning activities for students. 
Interactive technologies allow students to build new knowledge by doing, receiving feedback, and 
refining their understanding. Technologies may also help students to acquire more information, 
visualize difficult-to-understand concepts, and advance understanding. Immersion resources, thus, 
provide a means to extend, supplement, or enhance the state’s curriculum. 

In addition to instructional resources, technology immersion calls for online formative assessments as 
tools for teachers to diagnose students’ strengths and needs and assess progress toward curricular 
mastery. Teachers in each school will have access to a commercial assessment system with core 
content assessments along with access to the online Texas Mathematics Diagnostic System (TMDS). 
As a whole, electronic assessments yield more timely and meaningful data to inform decision making 
and shape students’ learning opportunities. 

Most importantly, implementing new resources and aligning them with existing curricula is expected 
to modify existing instructional practices and change the status quo. Yet, as others have pointed out, 
the availability of wireless laptops and digital resources may not improve student learning and 
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achievement if teachers fail to use resources or simply provide the same kinds of lessons and 
assignments electronically instead of using new technologies to transform students’ learning 
experiences (Means, Haertel, & Moses, 2003). Recognizing this reality, technology immersion 
includes components designed to support educators on their journey toward technology-infused 
schools and classrooms. 

Theoretical Framework for Technology Immersion 

The Theoretical Framework for Technology Immersion guides the evaluation (see Figure 1). The 
experimental design, as illustrated in the framework, allows an estimate of the effects of the 
intervention, which is the difference between the experimental and control groups. The framework 
also postulates a linear sequence of causal relationships. Program implementation comes first. 
Experimental schools are to he “immersed” in technology through the introduction of technology 
immersion components. The quality of implementation reflects the robustness of wireless laptop 
access for teachers and students, the adequacy of technical and pedagogical support services to 
maintain an immersed campus, the extent to which professional development supports curricular 
integration of technology, and how well curricular resources and assessments are used. Given quality 
implementation, we expect school-level improvements in measures of classroom technology 
integration, technical support, innovative culture, and parent and community support. Leadership and 
system support drives progress toward full immersion. 

An improved school environment for technology should then lead to teachers who have greater 
technology proficiency, use technology more often for their own professional productivity, collaborate 
more with their peers, have students use technology more and in new ways in their classrooms, and 
use laptops and digital resources to increase the intellectual challenge of lessons. In turn, these 
improved school and classroom conditions should lead students to greater technology proficiency, 
more opportunities for peer collaboration, greater personal self-direction, and stronger engagement in 
school and learning. Student mediating variables presumably will contribute to increased academic 
performance as measured by standardized test scores. In the framework, links are also shown between 
student achievement and student, family, and school characteristics, which exert their own influence 
on learning. 

The study’s theoretical framework has guided the evaluation design as well as the design of data 
collection procedures and measures. The research literature underpinning the framework is presented 
in sections to follow for school, teacher, and student variables. Although research on educational 
technology and one-to-one computing, in particular, has grown in recent years, there are still few large 
scale experimental studies or studies with well matched comparison groups. Thus, in some instances, 
evidence regarding variables is relatively strong, whereas in others areas evidence is weaker. 

School-Level Variables 

With immersion, technology resources are rooted in the school’s organizational and cultural 
environment — therefore, the intervention should change not just instruction and learning, but also the 
interactions between student and teacher, teacher and teacher, teacher and principal, and the school 
within the surrounding community (Dwyer, 1994). The evaluation of technology immersion, thus, 
must examine not only outcomes but also the factors that influence how and under what conditions 
technology is used to enhance students’ learning and achievement (Lowther, Ross, & Morrison, 2003). 
The sections below describe the key variables of interest at the school level: leadership and system 
support, classroom technology integration, technical support, innovative culture, and parent and 
community support. 
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Figure 1.1. Theoretical Framework for Technology Immersion 
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Leadership and System Support 

Over the past several decades, researchers have consistently concluded that school leadership is 
critical in developing and maintaining conditions that support school change and improvement (e.g., 
Hallinger & Heck, 1996 cited in Spillane, 2003; Rosenholtz, 1989). Similarly, administrative support 
is a major factor that influences technology integration (International Society for Technology in 
Education, 2002; Office of Technology Assessment, 1995). According to Johnston and Cooley, 
leaders in a technology-enhanced environment must he “champions of technology, teaching, learning, 
and students” (2000, p. 95). The principal, in particular, is a pivotal figure in effective technology 
implementation. The visionary principal is one who sees the integral relationship between technology 
and education, and marshals resources to help teachers master effective practice (Tinucci, 2000). 

In any systemic reform effort, such as technology immersion, a consistent vision and plan for change 
is also essential. Without systemic support, technology immersion may remain an untapped resource 
that has little impact on student learning (Means & Olson, 1994). Moreover, shared vision, or “huy- 
in,” is essential in moving complex systems towards substantive changes in instructional practice and 
student outcomes (Goertz, Floden, & O’Day, 1996). Careful planning is another prerequisite for 
effective technology implementation in schools, and when the campus plan is supported by district 
administrators, the effects are even greater (Cradler, 1992). Although Texas schools are incorporated 
into the local district technology plan, each school is expected to have its own vision for technology, 
usually developed in conjunction with the overall school improvement plan. 

Classroom Technology Integration and Technical Support 

As noted previously, Texas has strongly supported the infusion of technology into its schools. 
Consequently, at the start of this project, both treatment and control campuses will have an existing 
inventory of technology hardware, software, and educational programs. Furthermore, each of the 
participating middle school campuses is required to support students’ mastery of the Technology 
Applications, Texas Essential Knowledge and Skills (TEKS). Thus, schools may have technology 
applications classes (usually in a computer lab), or the technology TEKS may be integrated into the 
classroom context. Furthermore, prior to the project, districts and campuses had people such as 
technology coordinators and technical support personnel supporting technology. Given the existing 
context, school-level measures of the availability of resources for classroom technology integration 
and the extent of technical support are important. 

Innovative Culture 

Studies of the school change process suggest that technology immersion may both influence and be 
influenced by the school culture. In undertaking innovation, Senge stresses the importance of the 
organization’ s shared commitment to change and ability to build capacity for doing things in a new 
way (1999). Similarly, Fullan (1993) reports that some schools are more successful than others in 
sustaining innovation, and in more effective schools, change is a collective rather than an individual 
enterprise. In describing the three stages of the adoption of new technology. Chapman (1996) says 
that movement towards new ways of teaching and learning with technology is more significant if 
teachers are able to work collaboratively. Providing shared professional learning opportunities for 
teachers also is cited as a valuable means to stimulate innovative practice (Birman, Desimone, Porter, 
& Caret, 2000; Dibbon, 2003). Accordingly, we anticipate that the collective experiences of educators 
at immersed campuses will create a shared understanding of technology use and encourage integration 
efforts. Schools that begin the project as professional communities may advance at a more rapid pace 
(Fullan, 1999). 
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Parent and Community Support 

The local community may also influence the implementation of a comprehensive school reform effort. 
Its constituents may consist of parents, neighborhood residents, local professionals, city professionals, 
and elected school hoard officials. Educating and involving the community has been identified as a 
key component in ensuring successful change in educational practice (Goertz et ah, 1996). If parents 
and community members are “on the same page” as the school with regard to technology immersion, 
they can contribute the kind of support and resources required for changes in educational practices. At 
immersed campuses, community outreach may take many forms, such as participation on a technology 
committee, attendance at informational sessions or workshops, the dissemination of information 
through district and campus websites, or media releases to spread the word about technology 
immersion. In a one-to-one computing project, parents must also be partners in assuming 
responsibility for laptops used outside of the school. 

Teacher Variables 

Information from various research studies, especially those focusing on the implementation of 
technology initiatives, provide direction for the investigation of teacher variables. At the teacher level, 
it is hypothesized that technology immersion will lead to increased technology proficiency, greater use 
of technology for professional productivity, more frequent opportunities for students to use technology 
in classrooms, pedagogical changes such as increased technology integration and more learner- 
centered instruction. New technology also is expected to advance the intellectual demands of lessons 
and activities. Moreover, teachers in schools that are immersed in technology should begin to 
collaborate more often with their peers as they experiment with new instructional technologies and 
digital resources. 

Technology Proficiency 

A number of studies associate teachers’ technology proficiencies with technology implementation. 
Research indicates that educators must have a solid foundation of technology literacy before they can 
successfully integrate technology into the curriculum (Dusick, 1998; Goldsworthy, 2000). Hence, 
teachers must learn to use technology comfortably, efficiently, and effectively. Unfortunately, 
according to the American Council on Education (1999), many teachers lack the proficiencies and 
understanding to apply technology applications to instruction and learning. One study revealed that 
more than half of teachers felt only somewhat prepared to use technology for instruction, and more 
experienced teachers felt less prepared to use instructional technology than their more novice 
counterparts (NCES, 2000b). Surveys of Texas teachers showed improvements in educators’ 
proficiencies across a five-year period, although proficiency levels continued to be below targeted 
standards (Shapley et ah, 2002). 

Research also shows that teachers with strong computer skills use technology in greater numbers of 
ways and on a more regular basis, and these teachers are more likely to increase their technology-use 
frequency over time (Ronnkvist et ah, 2000). Teachers with the strongest technology proficiencies also 
use technology in more innovative ways for their content areas (Becker, 2000). Conversely, the 
primary reason teachers do not use technology in their classrooms appears to be a lack of experience 
with the technology (Rosen & Weil, 1995; Wenglinsky, 1998). 

Professional Productivity 

Skilled teachers also are more likely to use technology as a tool to enhance their own professional 
productivity, such as communicating with students and parents by email, creating electronic lesson 
plans, or accessing information from the Internet for lessons (Shapley et ah, 2002). Although 
researchers of one-to-one technology initiatives typically have not investigated teachers’ use of 



technology for professional productivity, it is important because Texas technology standards call for 
teachers to use technology effectively for communication as well as for acquiring, analyzing, and 
evaluating a variety of electronic information. For example, Texas teachers are increasingly expected 
to communicate by email, report attendance electronically, submit electronic lesson plans, post 
information on a campus website, and analyze and interpret electronic data from benchmark 
assessments. 

Collaboration 

Teachers also need time to discuss technology use with other teachers. Professional collaboration 
includes communicating with educators in similar situations and others who have experience with 
technology. Collaboration can be done in face-to-face meetings or by using technology such as email 
or videoconferencing. In the Maine laptop initiative, teachers found informal help from colleagues to 
be the most effective professional development activity. Technology, such as email, listservs, and 
websites, has also enabled Maine teachers to exchange information and stay in touch (MEPRI, 2003). 
According to Zhao and Frank, “Teachers who perceived pressure from colleagues were more likely to 
use computers for their own purposes, and teachers who received help from colleagues were more 
likely to use computers with their students” (2003, p. 825). 

Classroom Technology Use 

The introduction of one-to-one student access to laptops in classrooms also is expected to increase 
students’ opportunities for technology use. As noted previously, the link between teacher access to 
technology and increased classroom use is well documented. Teachers more often use computers and 
the Internet when these technologies are available in their classrooms rather than in other locations in 
the school (Becker, 2001; NCES, 2000b). Researchers for Maine’s one-to-one initiative found that 
teachers used technology more often, possessed a broad knowledge of technology resources, and were 
making progress in incorporating technology into practice (MEPRI, 2004). 

Technology Integration 

Providing more technology to support more frequent use is a start, but unless those technology 
resources are well integrated into instructional and learning experiences, the impact on student 
achievement is likely to be minimal. Some researchers say that helping teachers to understand new 
views of student learning and new models of teaching supported by technology is critically important 
in changing instructional practice (Bransford et ah, 2003; Johnston & Cooley, 2001). Rockman et al. 
(1998) demonstrated that many teachers view technology as an add-on or reward for students who 
finish their seatwork rather than an integral part of their pedagogical repertoire. 

Baker, Gearhart, and Herman’s (1994) five-year evaluation of Apple Classrooms of Tomorrow 
established a link between interactive technology use and changes in pedagogy. Specifically, teachers 
began to incorporate more collaborative work and fewer teacher-centered, lecture-oriented lessons in 
favor of student-centered ones. Other evaluators, including those of the Maine Eeaming Technology 
Initiative (MEPRI, 2003), have found evidence of teachers adjusting their pedagogical style, with 
students taking more responsibility for their own learning. Other researchers also found that classroom 
structures shifted from large group to students working independently or to more student-centered 
activities with one-to-one immersion (Rockman et al., 1998; Russell, Bebell, Cowan, & Corbelli, 
2002). 

Others suggest that teacher attitude is a factor that appears to affect the likelihood of technology 
integration and teachers’ perceived costs and benefits affect practices (Zhao & Frank, 2003). Dwyer, 
Ringstaff, and Sandholtz (1991) report that technology changes teacher beliefs as well as their 
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practice. Teachers’ beliefs and practices evolve along a continuum of technology integration that 
gradually leads to increasingly effective instructional practices. Movement from the entry phase to 
invention (technology-intensive environments) requires time and ongoing support. 

Intellectual Challenge 

The main benefit of technology immersion may be found in the opportunities available for more 
complex modes of teaching and learning. Researchers of technology-infused classrooms have 
demonstrated many positive effects of situated learning, especially those that bring real-life problems 
into the classroom (Temple & Rodero, 1995). Technology affords educators and students access to 
real-life problems or high-quality simulations of them. Technology also allows teachers to model 
learning strategies and allows individual learners to approach a task in different ways using different 
strategies (Goldman, Cole, & Syer, 1999; Many, Fyfe, Lewis, & Mitchell, 1996; Sulla, 1999). 

This view of technology’s potential for more advanced learning contrasts with evidence on prevailing 
classroom conditions. Even though three-quarters of teachers nationally report using computers or the 
Internet for instruction, most lessons fail to involve complex inquiries, explorations, or problem- 
solving activities (Doherty & Orlofsky, 2001). Similarly, Texas students and teachers mainly use 
technology at a basic level. According to statewide surveys, teachers most frequently use technology 
for basic tasks such as creating instructional materials, communication, and Internet research on a 
topic (Shapley et ah, 2002). 

Student Variables 

Over the past decade, a growing body of research has pointed to positive effects of technology on 
students’ skills, learning, and achievement. For this study, we look to evidence from studies of 
technology in general as well as more recent research on one-to-one technology initiatives. In the 
research literature, studies assert that technology fosters positive student effects for technology use, 
technical proficiencies, motivation and engagement, intellectually challenging schoolwork, self- 
direction, and to a lesser extent, academic achievement. Although many studies fail to meet rigorous 
research standards, the cumulative amount of evidence suggests areas of importance for scientific 
investigation. 

Technology Use 

One-to-one access to technology, not surprisingly, leads to students’ increased technology use. 
Russell, Bebell, & Higgins (n.d.) found that technology is used more often for instructional and 
learning purposes in one-to-one classrooms. In addition, several studies have shown that students 
involved in ubiquitous technology programs tend to use technology more often outside of the 
classroom as well. For instance, Russell et al. (n.d.) found that students in one-to-one classrooms used 
computers at home more frequently for academic purposes. Furthermore, Baldwin (1999) found that 
students spend less time watching television and more time on homework after they were provided 
with laptop computers. Rockman (2003) reports that laptops offer students the tools needed to access, 
manipulate, and organize information. Moreover, they provide a means of “closing the digital divide” 
between more advantaged students who have access to computers and the Internet at home and those 
without technology outside of school. 

Technology Proficiency 

Researchers also report that students’ technology proficiencies increase with ubiquitous technology. 
Rockman et al. (1998) found that laptop students considered themselves more proficient users of 
Word, Excel, PowerPoint, the Internet, email, and CD-ROMS. Similarly, Fowther, Ross, and 
Morrison (2001) indicated that fifth and sixth graders who were provided with laptop computers 
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reported increased computer skills and were better able to do Internet research. In another study, 
Schaumburg (2001) found that German high school students with laptops made greater gains than 
comparison students on measures of technology literacy, such as knowledge of hardware and the 
operating system, productivity tools, and Internet use. 

Motivation and Engagement 

Numerous studies have reported links between one-to-one technology and increased student 
engagement (MEPRI, 2003; Rockman et al., 1998; Russell et al., n.d.; Woodul, Vitale, & Scott, 2000). 
Evaluators of the Maine Eearning Technology Initiative found that students find school and learning 
more interesting and prefer using laptops for most school-related tasks (MEPRI, 2003). The five-year 
ACOT evaluation established a link between interactive technology use and student attitudes. The 
research team observed students voluntarily using time outside of school to work on technology-based 
projects, and students often initiated their own computer-related projects (Baker et al., 1994). 

Several researchers have focused on the link between technology access and use and student behavior. 
Barron, Hogarty, Kromery, and Eenkway (1999) examined the impact of the number of computers in 
use per student on student conduct in schools throughout Elorida. As the number of computers 
increased, middle schools experienced fewer conduct violations and disciplinary actions. Aeby, 
Powell, and Carpenter- Aeby (1999-2000) examined the effects of a computerized curriculum upon the 
psychosocial and academic outcomes of students identified as chronically disruptive. Students’ self- 
esteem improved through use of the curriculum. An evaluation of the North Carolina Eaptop 
Notebook Project revealed a strong correlation between computer use and improved attendance. 
Students participating in the laptop program had fewer absences and late arrivals as compared to non- 
participants (Stevenson, 1998). In Henrico County Public Schools in Virginia, the research team 
uncovered preliminary evidence showing increased student motivation, engagement, interest, and self- 
directed learning (Zucker & McGee, 2005). Other notable studies have also demonstrated a decrease 
in discipline problems linked to one-to-one computing (Baldwin, 1999; MEPRI, 2003). 

inteiiectuai Work 

The most common student technology use currently focuses on productivity tools, research, and drill 
and practice activities. Activities that involve higher-order thinking and collaboration, such as 
technology-based projects, multimedia authoring, problem solving using spreadsheets or databases, 
and correspondence with experts, remain less common (Becker, 1999, 2001; Denton et al., 2001; 
NCES, 2000b). Despite prevalent practice, “At its best, technology can facilitate deep exploration and 
integration of information, high-level thinking, and profound engagement by allowing students to 
design, explore, experiment, access information, and model complex phenomena,” note Goldman, 
Cole, and Syer (1999). Additionally, technology enables students to have increased access to and use 
of a wide range of information, allowing for greater inquiry and investigation, exposure to places and 
resources beyond the classroom, and development of a stronger knowledge base (CEO Eorum, 2001; 
Johnston & Cooley, 2001). 

These new circumstances and opportunities — not the technology on its own — can have a direct and 
meaningful impact on student achievement. Several studies, including Baker et al.’s (1994) five-year 
evaluation of Apple Classrooms of Tomorrow and Hopson, Simms, and Knezek’s (2002) study of 
technology integration, have established a tentative link between interactive technologies and higher 
level reasoning and problem solving. Moreover, many new interactive technologies allow students to 
build knowledge by doing, receiving feedback, and continually refining their understanding (Barron et 
al., 1999; Bereiter & Scardamalia, 1993). Technology also provides a medium for bringing real-world 
problems into the classroom for students to explore and solve. Students who have worked with the 
Jasper Woodbury Problem Solving Series have shown gains in mathematical problem solving. 
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communication abilities, and attitudes toward mathematics (e.g., Cognition and Technology Group at 
Vanderbilt, 1997). 

Self-Directed Learning 

Several researchers also have linked technology use to greater student self-direction of learning. This 
link is based on the premise that working one-to-one with technology allows hands-on, self-directed 
experience, as learners work independently much of the time. The theory of self-regulation assumes 
that a learner who knows how to be self-directed and independent will be more successful than one 
who is highly dependent on structured guidance. The teacher’s role is to scaffold learning by making 
learning more tangible and by modeling learning strategies (Bolhuis, 1996; Corno, 1992; Leal, 1993). 
Since self-directed learners are responsible owners and managers of their own learning process, 
control shifts, over time, from teachers to learners (Garrison, 1997). 

Self-regulated or self-directed learning strategies enable learners to solve problems in new domains of 
inquiry (Ertmer & Newby, 1996; Morrow, Sharkey, & Firestone, 1993). Another key aspect of self- 
directed learning is that it seeks to bridge the gap between academic knowledge and real-world 
problems (Bolhuis, 1996; Temple & Rodero, 1995). Some studies show that students who work in a 
self-directed context are more productive. For instance, when writers are allowed to choose their own 
topics, they write more often and they write longer pieces (Morrow et al., 1993). Raghavan, Sartoris, 
and Glaser (1997) linked computer-supported, engaged learning to self-directed learning, problem 
solving, and higher order thinking skills in middle school science students. In addition to taking more 
responsibility for their learning, students displayed more competence in complex problem-solving 
strategies. 

Academic Achievement 

The ultimate goal of technology immersion is to increase the academic progress of students. This 
study is important because no large-scale, controlled studies have measured the impact of one-to-one 
learning on student achievement. Available evidence on the effects of laptops on student achievement 
comes from a few studies that have made comparisons between student groups with and without 
technology. Findings, although limited, have generally been positive. 

The strongest evidence on the effect of laptops on achievement is in the area of writing. Fowther, 
Ross, and Morrison (2001, 2003) reported highly significant effects favoring sixth- and seventh-grade 
students with laptops over control students for dimensions of writing, such as ideas and content, 
organization, and style. In a less methodologically rigorous study, Rockman et al. (1999) also found 
that laptop students outscored non-laptop students on four measures of writing, including content; 
organization; language, voice, and style; and mechanics, conventions, and presentation. 

Some studies also have reported positive effects of one-to-one laptop access on students higher order 
problem solving (Fowther, Ross, & Morrison, 2003). The evaluation of the laptop project in Beaufort 
County, West Virginia focused on academic outcome measures, including a nationally standardized 
achievement test. Evaluators found that laptop students participating in the program for two years had 
higher language, reading, and mathematics scores than non-laptop students (Stevenson, 1998). 
However, since there was no statistical control for prior achievement, findings are in doubt. Certainly, 
additional research studies with experimental designs are needed to draw definitive conclusions about 
the effect of one-to-one initiatives on student achievement. 


12 



study Questions 


The evaluation of technology immersion employs a quasi-experimental research design with 44 
middle schools assigned to either treatment or control groups (22 schools in each). Researchers have 
posed six main research questions: 

• What are the haseline characteristics of participating schools? 

• How is technology immersion implemented? 

• What is the effect of technology immersion on schools? 

• What is the effect of technology immersion on teachers and teaching? 

• What is the effect of technology immersion on students and learning? and 

• Does technology immersion impact student achievement? 

This report concentrates on information gathered for the participating middle school campuses during 
the 2004-05 school year. Data collection involved a mix of qualitative and quantitative data sources. 
Researchers conducted site visits in each of the middle schools in fall 2004 and again in spring 2005. 
For this report, we include site-visit data from surveys of campus technology resources and 
observations in core-suhject classrooms. Additional measures, usually administered as pre-measures 
(fall 2004) and post-measures (spring 2005), include teacher and student surveys, and school and 
student data from the Texas Public Education Information Management System (PEIMS) and 
Academic Excellence Indicator System (AEIS). 

Organization of the Report 

Report sections are organized around findings relative to the study’s research questions. More 
specifically, key research areas that guided the synthesis of findings across the immersion and control 
sites are listed below. 

• Section 1, Introduction, provides background on the technology immersion project as well as 
the study’s theoretical framework and supporting research literature. The section also 
establishes the purpose for the study and the research questions addressed. 

• Section 2, Methodology, presents information on the evaluation design, characteristics of 
immersion and control schools, study limitations, study participants, data collection methods, 
and data analysis procedures. 

• Section 3, Baseline Characteristics of Participating Schools, describes patterns of technology 
access and use prior to the technology pilot as a way to establish the comparability of 
treatment and control schools. 

• Section 4, Technology Immersion Pilot — First-Year Implementation, includes information on 
progress made toward implementation in the first year. 

• Section 5, Effects of Technology Immersion on Schools and Teachers, presents findings on 
the impacts of immersion on school-level variables, including leadership and system support, 
classroom technology integration, technical support, innovative culture, and parent and 
community support. Additional findings are for teacher variables: technology proficiency, 
professional productivity, students’ use of technology, technology integration, collaboration 
with peers, and the intellectual challenge of instruction. 

• Section 6, Effects of Technology Immersion on Students and Learning, offers findings on the 
impacts of immersion on student mediating variables, including technology proficiency and 
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use, technical problems, opportunities for small-group work, school satisfaction, self-directed 
learning, and engagement. 

Section 7, Effect of Technology Immersion on Student Achievement, presents findings on the 
impact of technology immersion on sixth-grade student academic achievement as measured 
by TAKS reading and mathematics assessments. 

Section 8, Conclusions and Implications. This section summarizes the major findings from 
the study and discusses the implications. 



2. Methodology 


Evaluation Design 

The evaluation design is quasi-experimental with a carefully matched comparison group. The design 
aims to approximate a randomly assigned control group hy matching immersion schools with non- 
immersion schools possessing similar pre -program characteristics. For this study, interested districts 
and associated middle schools responded to a Request for Application (RFA) offered hy the Texas 
Education Agency (TEA) in spring 2004 to become technology immersion schools. Applicants to 
become Technology Immersion Pilot (TIP) sites had to meet eligibility requirements for Title II, Part 
D funds (i.e., high-need due to children from families with incomes below the poverty line, schools 
identified for improvement, or schools with substantial need for technology). 

Twenty-two technology immersion schools, selected through the competitive grant process, were 
matched by researchers with 22 control schools on key characteristics, including size, regional 
location, demographics, and student achievement. The TIP grants targeted high-need schools, thus 
nearly 70% of students in the study come from economically disadvantaged backgrounds, with many 
schools in rural or isolated locations. Students are ethnically diverse, roughly 56% Hispanic and 9% 
African American. 

The evaluation originally aimed for an experimental design with random assignment of 60 schools to 
treatment (n=30) and control (n=30) groups. Unfortunately, the recruitment of districts and related 
middle schools proved to be a major obstacle to the random assignment of schools due to funding 
restrictions (Title II, Part D) and the amount of available dollars ($14 million).' 

Treatment Sample 

In January 2004, the TEA released a REA for school districts to receive TIP grants for up to two 
middle schools. During this initial round, only 14 eligible districts applied — of these, 1 1 were small 
districts with one middle school campus, and 3 were larger districts with multiple middle schools. The 
majority of applicants had 300 or fewer students at their campus. The TEA held an external review of 
proposals in early May. Applications were scored by five readers and scores were rank ordered. 
Eollowing the external review, researchers and agency staff reviewed proposals to ensure that 
applications met the criteria established for technology immersion. 

At this point, researchers knew that a second round of applications would be necessary, so first-round 
choices concentrated on the selection of small schools for the treatment sample. In the selection 
process, researchers considered factors such as (a) REA rating scores, (b) district and campus size — 
small, mid-size, and large, (c) regional location (i.e.. Education Service Center region), (d) the 
proportion of economically disadvantaged and minority students, and (e) the percentage of students 
passing TAKS (all tests). Eive small middle schools (300 or fewer students) and one large middle 
school (900 students) were chosen in the first round of REAs. 

To increase the pool of middle schools for the evaluation study, a second REA for TIP grants (Round 
2) was released in late May 2004. Additional recruitment efforts were undertaken through phone. 


’ It was anticipated that grants would be awarded in amounts up to $350,000 to support technology immersion in grades 6 
through 8 middle schools. State-level statistics revealed an available pool of 486 middle schools. 
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email, mailings, and a videoconference for potential TIP grant applicants. In an effort to attract larger 
districts and middle schools, the RFA funding formula was modified to increase the amount of grant 
funds. The amount awarded to a participating campus was tied to campus enrollment: 350 students or 
less (up to $350,000), 351-600 students (up to $600,000), and greater than 600 students ($750,000). 
Non- funded applicants from Round 1 also were eligible to reapply and all but two districts did so. 

During the second round, 22 eligible schools applied. Comparable to Round 1, expert reviewers rated 
the 22 proposals, and 19 proposals with a score 85 or above were eligible for selection. The selection 
process for treatment schools mirrored the first round. Researchers considered proposal ratings, size, 
location, student diversity, and academic achievement. Decisions were strongly influenced by the need 
for geographic distribution and the availability of comparable schools for the control group pool. Of 
the 19 proposals, 16 middle schools were selected for the treatment group (immersed campuses). The 
3 non-selected campuses became part of the pool of middle schools available for the control group. 
These campuses had proposal scores comparable to selected campuses (at least 85 points) — however, 
they were not selected due to other considerations (i.e., geographic location and invoking a two- 
campus limit per district). 

Of the 22 treatment schools, 14 are in small, single middle school districts (enrolling less than 3,000 
students), 7 are schools in large districts (enrolling 10,000 or more students and having multiple 
middle schools), and 1 is a campus charter school in a large urban district (with more than 200,000 
students). In sum, the originally envisioned random assignment of schools to experimental and control 
groups was not possible since the applicant pool did not include enough schools. Instead, researchers 
have used a matched control group research design, with experimental and control group campuses 
matched on key demographic and achievement variables. The sample size is considered adequate to 
detect a small effect size (.25 or larger; Cohen, 1988). 

Control Sample 

The selection of control campuses involved several steps. First, in order to increase the available pool 
of middle schools that would be eligible to receive federal funds for participation in the study, 
researchers generated a pool of eligible grades 6-8 campuses from a list of districts that had previously 
received TARGET grants (Technology Applications Readiness Grants for Empowering Texas), 
competitive grants funded with Title II, Part D funds. Thus, the control pool now included 251 middle 
school campuses in districts receiving TARGET funds and 6 schools that had applied for and earned 
proposal rating scores that qualified them for TIP funds (3 each in the two rounds). Of these campuses, 
63 had 600 or fewer students and 194 had 601 or more. 

As a next step, researchers identified middle schools that matched treatment campuses as nearly as 
possible on factors, including (a) district and campus size, (b) regional location, (c) the proportion of 
economically disadvantaged and minority students, (d) percentage of students passing all TAKS tests, 
and (e) the gaps between the percentage of White students and African American and Hispanic 
students passing TAKS (all tests). Selection involved the use of 5'P5S® statistical software procedures 
to establish parameters around each variable of interest and the creation of a computer-generated list of 
“best matches” for each treatment school. In addition, grant specifications required large districts with 
multiple middle schools to provide access to control campuses within their own district, assuming a 
comparable school was available, so these schools were added to the list. 

The final selection process for the control group involved a review of the matched list by a team of six 
researchers to identify the optimal control school for each treatment school. Additional schools were 
selected as alternates in the case that a selected control site declined the invitation to participate in the 
study. This selection process yielded 22 control group schools including controls for 8 campuses that 
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came from within the same districts as the treatment schools and controls for 14 campuses from 
closely matched single, middle school districts. Similar to the experimental group, the 22 control 
schools included 13 in small, single middle school districts (enrolling less than 3,000 students), 8 
schools in large districts (enrolling 10,000 or more students and having multiple middle schools), and 
1 campus charter school in a large urban district (with more than 200,000 students). Each control 
school received $50,000 for study participation, with 25% of funds earmarked for professional 
development as required hy Title II, Part D guidelines. 


Characteristics of Participating Schoois 

The schools participating in the study are compared in Table 2.1 by assignment (treatment and control) 
and student enrollments. The distribution of middle schools across campus and district enrollment 
categories shows the comparability of treatment and control groups. For both groups, middle schools 
are typically small (enrolling 600 students or less), and they are located either in small or very small 
districts (enrolling 2,999 students or less) or large districts (enrolling 10,000 students or more). 


Table 2.1. Campus and District Enrollment by Comparison Group 



Immersion N=22 

Control N=22 

Number of students 

Number 

Percent 

Number 

Percent 

Campus 

300 or less 

12 

54.5 

12 

54.5 

301-600 

5 

22.7 

5 

22.7 

601 or more 

5 

22.7 

5 

22.7 

District 

999 or less 

8 

36.4 

8 

36.4 

1,000-2,999 

6 

27.3 

5 

22.7 

3,000-9,999 

0 

0.0 

0 

0.0 

10,000 or more 

8 

36.4 

9 

40.9 


Information in Table 2.2 compares the baseline characteristics of immersion and control schools. 
Results for t-tests show that the percentages of economically disadvantaged, minority, English as a 
second language (ESE), and special education students are statistically equivalent across the treatment 
and control schools. Eikewise, results for student enrollment, mobility, and TAKS passing rates for all 
tests taken also showed no significant differences. Consequently, the treatment and control schools are 
sufficiently well matched on key demographic and academic performance measures. Moreover, both 
treatment and control samples include a range of campus and district enrollments and schools from 
diverse regions of the state. In these respects, the sample selection process and matching procedures 
appear to have produced a baseline sample of schools with good internal validity, in that there are no 
large, statistically significant treatment-control differences. Still, the tendency for immersion schools 
to enroll greater proportions of minority and economically disadvantaged students may be important 
considering known links between disadvantaged status and lower achievement. 
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Table 2.2. Comparison of Baseline Characteristics: Technology Immersion {N=22) 
and Control Schools (A/=22) 

I 95% Confidence Interval for Difference 


Variable 

Condition 

Mean 

SD 

Lower 

Upper 

f(43) 

Enrollment 

Immersion 

394.0 

351.6 

-185.6 

261.3 

0.34 


Control 

432.0 

382.2 




Economic disadvantage (%) 

Immersion 

71.3 

17.3 

-19.2 

2.6 

-1.54 


Control 

63.0 

18.5 




Minority (%) 

Immersion 

69.1 

28.1 

-24.3 

9.5 

-0.89 


Control 

61.7 

27.4 




ESL (%) 

Immersion 

12.9 

17.0 

-15.4 

1.6 

-1.64 


Control 

6.1 

9.7 




Special education (%) 

Immersion 

14.6 

5.4 

-2.4 

3.7 

0.45 


Control 

15.3 

4.5 




Student mobility (%) 

Immersion 

15.7 

4.5 

-2.3 

4.2 

0.60 


Control 

16.7 

6.1 




TAKS 2004, Passing All (%) 

Immersion 

51.1 

16.5 

-6.8 

10.9 

0.47 


Control 

53.1 

12.3 




TAKS 2003, Passing All (%) 

Immersion 

65.5 

11.3 

-5.6 

8.6 

0.42 


Control 

67.0 

12.1 





Source: Texas Education Agency AEIS reports 2004 

Note. TAKS=Texas Assessment of Knowledge and Skills. Differences between groups are statistically insignificant. 


Table 2.3 provides campus-level data for each of the 44 schools recruited for the study. Again, data in 
the table show that the treatment and control schools are reasonably well matched on baseline 
characteristics. Middle schools are highly concentrated in rural and very small districts across the 
state. Still, about a third of the districts and schools are in large cities or suburban locations in or 
around cities. The sample also includes campus charter schools (one each for the treatment and control 
group) located in a major urban district. 

The primary limitation of the study is external validity — the extent to which the results of an 
experiment can be generalized from the specific sample to the general population. Schools eligible to 
become part of the treatment group were limited to those serving children from families living in 
poverty^ and grades 6 to 8 middle schools. Only schools that applied for the grant, and submitted 
applications that met a threshold of quality, were eligible for consideration. Due to these restrictions, 
the treatment group is not representative of the average middle school in Texas. 

The majority of students in the sample are economically disadvantaged. The percentage of sample 
students who qualify for federal free or reduced-price lunch exceeds the state average for middle 
schools (67% vs. 51%). The sample also is substantially more Hispanic and less white and African 
American than state middle-school students as a whole. Overall, about 56% of sample students are 
Hispanic compared to about 37% of Texas middle school students. Accordingly, compared to the 
state, the sample includes fewer African American students (9% vs. 14%) and fewer white students 
(34% vs. 46%). 


2 

Federal definition used: 27% of population or more than 2,500 people living below poverty line. 
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Table 2.3. Characteristics of Technology Immersion and Matched Control Schools 
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The sample schools also differ structurally from Texas middle schools as a whole. Middle schools in 
Texas typically enroll more students (609, on average, vs. 413 in sample schools) and are concentrated 
in larger districts (27,000 students enrolled, on average, vs. 18,737 in sample schools). Thus, 
compared to the state overall, sample schools and the districts they reside in are smaller and serve 
more economically disadvantaged and Hispanic students. Differences almost certainly reflect funding 
restrictions (Title II, Part D) and the amount of available funds per grant. The maximum amount 
($750,000) fell well short of dollars required to support one-to-one technology in larger middle 
schools. 


Participants 


Students 

Table 2.4 shows that three groups or cohorts of students will be followed in this study, with Cohort 1 
followed for four years. Cohort 2 for three years, and Cohort 3 for two years. Data collection activities 
in 2004-05 centered on the initial sixth-grade cohort, which included a total of 5,564 students (2,570 
enrolled at immersed campuses and 2,994 students at control campuses). 


Table 2.4. Student Cohorts by School Year and Grade 




Middle School 


High School 

Year 

Grade 6 

Grade 7 

Grade 8 

Grade 9 

2004-05 

Cohort 1 




2005-06 

Cohort 2 

Cohort 1 



2006-07 

Cohort 3 

Cohort 2 

Cohort 1 


2007-08 


Cohort 3 

Cohort 2 

Cohort 1 


Similar to sample schools as a whole, about three-fourths of sixth-graders are economically 
disadvantaged (see Table 2.5). Comparison groups also have nearly equal proportions of minority 
students and female and male students. The main difference between groups is the greater proportion 
of limited English proficient (LEP) students in immersed schools. 

Table 2.5. Demographic Characteristics of Sixth Grade Students: 

2004-05 



Immersion 

Control 

Characteristic 

N 

Percent 

N 

Percent 

Enrollment 

2,570 

— 

2,994 

— 

Economic disadvantage 

1.971 

76.7 

2,243 

74.9 

Ethnicity 

African American 

303 

11.8 

339 

11.3 

Hispanic 

1,708 

66.5 

1,916 

64.0 

White 

523 

20.4 

716 

23.9 

Other 

36 

1.4 

23 

0.8 

Limited English proficient 

522 

21.5 

474 

15.8 

Gender 

Eemale 

1,209 

47.0 

1,452 

48.5 

Male 

1,361 

53.0 

1,542 

51.5 


21 





Teachers 


During the 2004-05 school year, 1,304 teachers participated in the study (622 at immersed campuses 
and 682 at control campuses). Teachers in comparison groups are remarkably similar in terms of 
gender, ethnicity, advanced degrees, and average teaching experience. 


Table 2.6. Demographic Characteristics of Teachers: 2004-05 


Characteristic 

Immersion 

Control 

Number of teachers 

622 

682 

% Female 

65.4 

68.8 

% Minority 

42.4 

35.3 

% African American 

7.8 

7.5 

% Hispanic 

32.2 

26.3 

% White 

57.6 

64.7 

% with no degree 

0.0 

2.0 

% with advanced degree 

21.7 

22.2 

Average years of teaching experience 

10.9 

11.4 


Note. There were 1,304 teachers in 2004-05. 


Data Collection 

Data collection began in August 2004. As Table 2.7 illustrates, researchers conducted site visits in 
each of the 44 middle schools in fall 2004 and spring 2005. For this report, we concentrate on data 
gathered through observations in a sample of sixth-grade classrooms (English/language arts, 
mathematics, social studies, and science). Additional measures, administered as pre- and post- 
measures in fall and spring, include a Campus Technology Inventory completed by the campus 
technology coordinator, teacher online surveys, and student paper-and-pencil surveys. Additionally, 
we gathered school and student demographic, attendance, and achievement data from the Texas Public 
Education Information Management System (PEIMS) and Academic Excellence Indicator System 
(AEIS). In spring 2005, individual middle schools submitted student-level data on disciplinary actions. 


Table 2.7. Time Frame for Data Collection: 2004-05 



Fall 2004 

Spring 2005 

Site Visits 

Building walkthrough 

X 

X 

Classroom observations (6th grade) 

X 

X 

Other Measures 

Campus Technology Inventory 

X 

X 

Teacher Technology Survey (all teachers) 

X 

X 

Student Technology Survey (6th grade) 

X 

X 

Style of Learning Inventory (6th grade) 

X 

X 

Student Performance 

Texas Assessment of Academic Skills (TAKS)“ 

X 

X 

Attendance^ 

X 

X 

Discipline referrals and placements 

X 


Note. Data collection for 22 immersion and 22 control schools. 
TAKS and attendance data also were collected for spring 2003. 
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Measures 


Instruments selected to measure mediating and outcome variables, as described below, included 
surveys, a technology inventory, classroom observation form, and student performance measures. 
Appendix A provides information on survey items and the reliability of scale scores. 

Teacher Survey 

Immersion and control teachers (grades 6 to 8) completed an online technology survey in fall 2004 
(September to October) and spring 2005 (April to May). The survey included items related to school 
technology and teachers’ technology proficiency and use. In fall, 1,271 teachers completed the survey 
(97% of all teachers, 97% of immersed teachers, and 98% of control teachers). In spring, 1,144 
teachers (about 88% of all teachers, 87% of immersed teachers, and 88% of control teachers) 
completed the survey. 

School mediating variables. Teachers responded to 33 items pertaining to their perceptions of school 
technology. They rated the strength of their agreement with statements on a 5-point scale ranging from 
1 {strongly disagree) to 5 {strongly agree). Item analysis using maximum likelihood factor analysis 
with Varimax rotation revealed five distinct factors, including Leadership and System Support (12 
items). Classroom Technology Integration (4 items). Technical Support (5 items). Innovative Culture 
(4 items), and Parent and Community Support (2 items). Cronbach’s alpha coefficients (measures of 
internal consistency reliability) for school-level factors range from 0.67 to 0.91. 

Teacher mediating variables. Surveys completed by teachers also included measures of teacher 
mediating variables. Teachers responded to items pertaining to their perceptions, including perceptions 
of Technology Proficiency (27 items). Professional Productivity (17 items). Students’ Technology Use 
(17 items), and Collaboration (11 items related to teacher interactions with colleagues). In addition, 
confirmatory analysis of items adapted from the Levels of Technology Implementation (LoTi) 
Questionnaire (Moersch, 2001) showed reasonable fit indices for a model having Technology 
Integration (10 items), Learner-Centered Instruction (4 items), and Resistance to Integration (3 items). 
Cronbach’s alpha coefficients for teacher-level variables ranged from 0.70 (the 3-item Resistance to 
Integration scale) to 0.97 (the 27-item Technology Proficiency scale). 

For Technology Proficiency items, teachers indicated their skill level on a 7-point scale with 1 and 2 
indicating low proficiency {not true of me now), 3, 4, and 5 indicating moderate proficiency 
{somewhat true of me now), and 6 and 7 indicating proficiency {very true of me now). Measures of 
integration — Technology Integration, Learner-Centered Instruction, and Resistance to Integration — 
also involved a 7-point scale ranging from 1 {not true of me now) to 7 {very true of me now). For 
Professional Productivity, Students’ Technology Use, and Collaboration, teachers used a 5-point scale 
to rate the frequency of activities or interactions: 1 {never), 2 {rarely-e.g., a few times a year), 3 
{sometimes-e.g., once or twice a month), 4 {often-e.g., once or twice a week), and 5 {almost daily). 

Student Surveys 

Sixth-grade students completed paper-and-pencil questionnaires measuring their technology 
proficiency and use in fall 2004 (September to October) and spring 2005 (April to May). In addition, 
they completed the Style of Learning Inventory (SLI). The SLI is a measure of student self-directed 
learning (i.e., self-generated behaviors oriented toward the attainment of learning goals). The SLI, a 
paper-and-pencil questionnaire, was administered as a baseline measure in fall and as a post-measure 
in spring. 
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Technology survey. In fall, 4,824 sixth-grade students (87%) completed the survey. Respondents 
included 2,319 treatment students (90%) and 2,505 control students (84%). In spring, 4,538 students 
(82%) completed the survey, with 2,053 treatment students (80%) and 2,485 control students (83%). 
Survey items measured students’ Technology Proficiency (22 items). Technology Use in School (12 
items). Technology Use in English Language Arts, Math, Science, and Social Studies (12 items for 
each subject). Technical Problems (6 items), Small-Group Work (9 items), and School Satisfaction (6 
items). Cronbach’s alpha coefficients for student-level factors range from 0.76 to 0.94. 

As a measure of Technology Proficiency, students indicated how well they could use various 
technology applications on a 5-point scale: 1 (/ can do this not at all or barely), 2 (/ can do this with 
some difficulty), 3 (I can do this fairly well), 4 (I can do this very well), and 5 (/ can do this extremely 
well). For measures of Technology Use, Technical Problems, and Small-Group Work, students used a 
5-point scale to rate the frequency of activities or interactions: 1 {never), 2 (rarely-e.g., a few times a 
year), 3 (sometimes— e.g., once or twice a month), 4 (often-e.g., once or twice a week), and 5 (almost 
daily). Students rated school satisfaction items on a 5-point agreement scale ranging from 1 (strongly 
disagree) to 5 (strongly agree). 

Style of Learning Inventory. Sixth-grade students also completed the Style of Learning Inventory 
(developed by Metiri Group, 2004). In fall 2004, a total of 4,584 students (82%) completed the SLI as 
a baseline measure. Respondents included 2,142 treatment (83%) and 2,442 control students (82%). A 
total of 4,294 students (77%) completed the SLI again in spring 2005, including 2,174 treatment 
(85%) and 2,120 control students (71%). The SLI is a 48-item survey based on a model of self- 
regulated learning developed by Schunk and Zimmerman in 1998.^ The items on the SLI are 
categorized into 12 scales and three groupings. The three grouping and related scales are listed below. 

■ Forethought is defined as influential processes and beliefs that precede efforts to learn (goal 
setting, strategic planning; self-efficacy beliefs; goal orientation; and intrinsic interest), 

■ Performance/Volition control refers to processes that occur during learning efforts and affect 
concentration and performance (attention focusing, self-instruction, imagery; self-monitoring; 
and help seeking), and 

■ Self-reflection involves processes that occur after learning efforts and influence a learner’ s 
reaction to that experience. Since the learning process is cyclical, these processes will in turn 
influence forethought regarding subsequent learning efforts (self evaluation, attributions, self 
reactions, and adaptivity). 

Students rated statements regarding their personal self-direction on a 7-point scale, ranging from 1 
(completely false) to 7 (completely true). A factor analysis of SLI data for sixth graders was conducted 
in fall. Results revealed very low internal consistency of the SLI scales and groupings. Results showed 
that none of the groups of items or scales had sufficient reliability, with Cronbach’s Alpha coefficients 
ranging from 0.18 (Attributions) to 0.52 (Goal Setting, Strategic Planning and Help Seeking). 
Consequently, analyses are limited to the SLI total score, which has a reliability coefficient of 0.88. 

Campus Technology Inventory 

The instructional technology coordinator for each campus completed a Campus Technology Inventory 
in fall and spring to document the availability of technology. Survey items addressed technology 
access in the school and classrooms as well as technical and pedagogical support. In both fall and 
spring, coordinators for 22 immersion and 21 control schools completed inventories. 


2 

Schunk, D., & Zimmerman, B. (1998). Self-Regulated Learning from Teaching to Self-Reflective Practice. NY : Gilford 
Press. 
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Observation of Teaching and Learning 

Researchers conducted classroom observations in a sample of sixth-grade classrooms (reading/English 
language arts, mathematics, social studies, and science). The Observation of Teaching and Learning 
(OTL) form allows the documentation of basic descriptive information (e.g., number of students, 
content area), technology access and use (i.e., technology available and used by the teacher and 
students), and classroom environment (i.e., organization and management). In addition, researchers 
used time-interval ratings to record information in six areas: class organization (e.g., individual 
students, pairs, small groups, whole group), teacher activities (e.g., directing, guiding substantive 
discussion), teacher’s technology use (e.g., peripherals, presentation software), student activities (e.g., 
listening, learning facts, definitions, algorithms), students’ technology use (e.g., express themselves in 
writing, learn/practice skills), and student engagement (rated on a 5-point scale from low engagement 
to high engagement). Observers made the first rating after observing for 5 minutes, then made a rating 
every 10 minutes. During the observation, observers also recorded descriptive notes on the lesson 
objectives, teachers’ questioning strategies (lower or higher order), and class activities. Observations 
lasted about 45 minutes. 

After the observation, and based on time-interval ratings and descriptive notes, observers rated the 
intellectual challenge of classroom work. Relying on rubrics developed by Newmann, Secada, and 
Wehlage (1995), observers rated four dimensions of intellectual challenge on a 5-point scale: 
Construction of Knowledge — Higher Order Thinking, Disciplined Inquiry — Deep Knowledge, 
Disciplined Inquiry — Substantive Conversation, and Value Beyond School — Connections to the 
World Beyond the Classroom. An aggregate score across the four scales is used as an overall measure 
of the Intellectual Challenge. 

Number of observations. During fall 2004, researchers conducted classroom observations at 22 
middle schools (11 treatment and 11 control). In spring, we expanded observations to all 44 of the 
middle schools. In fall, researchers observed 142 classrooms (69 treatment and 73 control). During 
spring site visits, we conducted follow up observations, when possible, in the same classrooms. Data 
collection in spring involved observations in 235 classrooms (117 treatment and 118 control). At small 
campuses, researchers observed all sixth-grade core-subject teachers. For larger campuses, we 
observed at least eight classrooms (about 75% of sixth-grade teachers). 

Training procedures. Prior to fall site visits, researchers participated in a two-day training event. 
Training activities informed data collectors about the research design, aspects of technology 
immersion, data collection protocols, effective interview and focus group techniques, and classroom 
observation procedures. Approximately eight hours were devoted to the establishment of inter-rater 
agreement on the Observation of Teaching and Learning (OTL) form. During observation training, 
raters first reviewed background information and individual item and code definitions in the OTL 
manual. Raters next viewed a video in which a classroom teacher used technology as part of a lesson. 
The trainer stopped raters at 10-minute intervals to record ratings, discuss the extent of agreement or 
disagreement, and resolve misunderstandings. This process was repeated for an additional classroom 
video. 

To further enhance inter-rater agreement, raters were paired for observations in classrooms during the 
initial site visit at one immersion and one control school. Following classroom observations, raters 
again discussed assigned ratings and resolved disagreements. Classroom observations used for training 
purposes at these middle schools were excluded from statistical analyses. For subsequent site visits to 
other middle schools, observers were paired for about 25% of classroom observations. Overlapping 
observations allowed the calculation of observer reliability (i.e., the percentage of exact agreement on 
ratings from paired observations). Additionally, paired observations supported the use of Many -facets 
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Rasch Analysis (MFRA) to adjust scale scores on the Intellectual Challenge factor for the relative 
difficulty of each scale and the relative severity (or leniency) of each observer. 

In spring 2005, an additional one-day training event preceded site visits. The day included an 
overview of project activities, a review of information on treatment and control sites, spring data 
collection protocols, interview and focus group techniques, and classroom observation procedures. 
Approximately half of the day focused on improving inter-rater agreement on the OTL form. Similar 
to fall training, raters reviewed individual item and code definitions in the OTL manual, viewed and 
rated a video in which a classroom teacher used technology as part of the lesson, and discussed their 
agreement or disagreement and resolved misunderstandings. For subsequent site visits, observers were 
paired for about 25% of classroom observations. Following observations, raters discussed their level 
of agreement but did not change assigned ratings. Overlapping observations allowed the calculation of 
inter-rater agreement and the use of MFRA adjustments. 

Inter-rater agreement. Inter-rater agreement has been established for the Intellectual Challenge 
component of the classroom observation instrument. For this element, observers used 5-point rating 
scales to measure students’ higher-order thinking, disciplined inquiry in the area of deep knowledge, 
disciplined inquiry in the area of substantive conversation, and connections to the world beyond the 
classroom (Newmann, Secada, & Wehgle, 1995). Observer reliability on these scales was measured by 
calculating the percentage of time observers agreed on ratings from paired observations. Analyses of 
observations from fall of 2004 indicate 78% inter-rater agreement across 36 teachers. Agreement 
reaches 98% when scale categories are allowed to vary by one scale point (on the 5-point scale). 
Analyses for spring data are currently underway. 

An overall measure of Intellectual Challenge for each teacher was constructed using Many-Facets 
Rasch Analysis (MFRA). The quality of instruction measure is an aggregate score across the four 
scales, and is adjusted for the relative difficulty of each scale and the relative severity (or leniency) of 
each observer. MFRA produces several fit statistics that can be used to measure each observer’ s 
intrarater reliability or internal consistency. One of these, observer infit, weights each standardized 
residual by its variance and is more sensitive to unexpected patterns of small residuals. A second 
statistic, observer outfit, is an unweighted mean-square residual sensitive to outlying residuals 
(Linacre, 2004). There is no fixed rule for setting upper and lower limits for theses fit statistics. In 
some instances “misfitting” raters (observers) have been defined as having either a mean-square infit 
or outfit statistic greater than 1.5 (Lunz, Wright, & Linacre, 1990). In other cases the range has been 
from 0.5 to 3.0 (Myford & Wolfe, 2000). 

For this study, we define a “misfitting” observer as one with either a mean-square infit or outfit 
statistic less than 0.5 or greater than 1.5. This defines “misfit” as less than 50% of the variance in 
ratings than is modeled (a muted pattern) and more than 50% of the variance than is modeled (a noisy 
pattern). MFRA analyses of the fall observation data resulted in observer infit values from 0.61 to 1.34 
and observer outfit values from 0.62 to 1.20. No unusual rating patterns appeared to be present in the 
observation data. There did not appear to be unpredicted or overly predictable ratings (Linacre, 1995). 

Texas Assessment of Knowledge and Skills (TAKS) 

The TAKS is Texas’ criterion-referenced assessment that annually measures students’ mastery of the 
state’s content standards. The TAKS assesses reading at grades 3 to 9; English language arts at grades 
10 and 11; writing at grades 4 and 7; mathematics at grades 3 to 11; science at grades 5, 10, and 11; 
and social studies at grades 8, 10, and 11. Stringent quality control measures are applied at all stages 
of test administration, scanning, scoring, and reporting. Internal consistency reliabilities for TAKS 
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assessments are in the high .80s to low .90s range. Evidence also supports the content, construct, and 
criterion-related validity of TAKS assessments."^ 

Cohort 1 sixth graders completed the TAKS reading and mathematics assessments as posttests in April 
2005. Students’ scores for April 2004 were collected as pretests. At grade 6, TAKS reading measures 
four objectives : understanding of culturally diverse written texts, knowledge of literary elements, use 
of strategies to analyze written texts, and application of critical-thinking skills. TAKS mathematics at 
grade 6 measures six ohjectives: numbers, operations, and quantitative reasoning; patterns, 
relationships, and algebraic reasoning; geometry and spatial reasoning; concepts and uses of 
measurement; probability and statistics; and mathematical processes and tools used in problem 
solving. 

School Attendance and Disciplinary Actions 

A post- measure of sixth graders’ attendance came from PEIMS data for the 2004-05 school year and 
data for the previous two school years (2002-03 and 2003-04) provided pre-measures. Additionally, 
individual campuses submitted data for disciplinary actions taken during the 2004-05 school year. 

Data files included three indicators: (a) the number of office referrals (number of times a student was 
referred to the office for disciplinary purposes), (b) student suspension (whether the student was ever 
suspended — in-school or out-of-school — during the year), and (c) placement in an alternative 
education setting (educational, disciplinary, or juvenile justice). 


Technical information is available on the Texas Education Agency website at http://www.tea.state.tx.us/student 
assessment/resources/techdig04/index.html. 
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3. Baselm^e Characteristics of Participating Schools 

The use of a quasi-experimental design requires researchers to demonstrate to the extent possible that 
any detected effects cannot he attributed to pre-existing differences between treatment and control 
campuses. Thus, relying on our theoretical framework, we have collected extensive baseline data on 
the characteristics of schools, teachers, and students. 


School Characteristics 

The measurement of school-level variables — technology access and support services, administrative 
leadership and system support, classroom technology integration, school culture, and parent and 
community support — provided one source of evidence on the equivalence of comparison groups. 

Technology Access 

To gauge existing levels of technology in schools, technology coordinators or specialists at each of the 
sites were asked to complete a Campus Technology Inventory. Information in Table 3.1 shows that in 
fall 2004 both immersion and control campuses had little technology in their classrooms, with an 
average of about 2 desktop computers in treatment and control classes. Classrooms in control schools 
were slightly more likely to have printers, whereas treatment classrooms more often had laptop 
computers. The greater prevalence of laptops likely reflects the fact that some treatment teachers had 
received their grant-funded laptops prior to fall data collection. Middle school classrooms seldom had 
other technology resources, such as Liquid Crystal Display (LCD) projectors, although equipment was 
often available for checkout in a library or media center. Control campuses had slightly more 
computer labs and more computers in their labs. Computer labs usually had printers but other 
technology resources in labs, similar to classrooms, were scarce. 


Table 3.1. Campus Technology Inventory (Mean Number) 


Items 

Immersion 

N=22 

Control 

N=21 

Instractional classrooms 

24.0 

21.4 

Desktop computers 

2.0 

2.1 

Laptop computers 

0.8 

0.6 

Printers 

0.7 

1.0 

LCD projectors 

0.3 

0.3 

Computer labs 

2.1 

2.5 

Desktop computers 

18.0 

20.9 

Laptop computers 

0.1 

0.4 

Printers 

1.1 

1.2 

LCD projectors 

0.3 

0.5 


Technical and Pedagogical Support 

Campus technology leaders also indicated the strength of their agreement with statements regarding 
the provision of technical and pedagogical support for technology on their campuses. Respondents 
rated statements on a 5-point scale ranging from (1) strongly disagree to (5) strongly agree. As a 
whole, control campuses reported slightly stronger technical support for technology at their schools in 
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fall 2004. Technology leaders at control campuses reported more timely hardware and software 
repairs, greater Internet access and download speeds, and more sufficient professional development. 
Adequate support for hardware issues appeared to he the greatest problem for all schools. 

School Technology 

Information on other school technology variables comes from the baseline survey of immersion and 
control teachers conducted online between September and October 2004. In addition to other areas of 
interest, teachers responded to items pertaining to their perceptions of school technology. Teachers 
rated the strength of their agreement with statements on a 5 -point scale ranging from 1 {strongly 
disagree) to 5 {strongly agree). Table 3.2 shows group differences for five distinct factors that 
emerged from a factor analysis of survey items. 


Table 3.2. Group Differences for School Technology Variables 


Variables 

Immersion 

Control 

f-value 

P 

Effect 

Size 

Mean 

SD 

Mean 

SD 

Leadership and System Support 

3.70 

0.61 

3.59 

0.69 

3.03 

.003* 

0.17 

Classroom Technology Integration 

3.24 

0.82 

3.23 

0.83 

0.12 

.902 

0.01 

Technical Support 

3.25 

0.77 

3.26 

0.78 

-0.33 

.739 

-0.02 

Innovative Culture 

3.70 

0.67 

3.70 

0.68 

0.02 

.984 

0.00 

Parent and Community Support 

3.41 

0.82 

3.39 

0.75 

0.37 

.710 

0.02 


Note. Immersion (n ranges from 569 to 579 teachers), Control (w ranges from 643 to 650 teachers). Scores range from 1 
(strongly disagree) to 5 (strongly agree). *Statistically significant difference. Effect size is Cohen’s d. 


Comparisons reveal that teachers in both immersion and control schools share similar views on their 
school’s educational technology. Only one statistically significant difference between groups was 
found, and this difference represented a small effect size {d - 0.17). Teachers on campuses that had 
received grants for technology immersion indicated that their school had stronger Leadership and 
System Support for technology. For example, teachers were more likely to believe “the principal is an 
effective leader for technology” and “our school has a well-developed technology plan that guides all 
technology integration efforts.” Since the teacher survey was conducted after middle school campuses 
received notification about the immersion grant award, this may have been a factor in teachers’ more 
positive perceptions of administrative support. Alternatively, since administrators at treatment schools 
and districts took the initiative to apply for grants, they may actually be stronger technology 
proponents. 

Teacher Characteristics 

We also collected extensive baseline data on teacher mediating variables through surveys of grades 6 
through 8 teachers. Comparisons in Table 3.3 reveal that immersion and control teachers reported 
comparable levels of Technology Proficiency, Students’ Technology Use (in their classrooms), and 
Collaboration (interactions with peers on technology issues). Two statistically significant differences 
between groups emerged, although the mean differences and effect sizes were small. First, teachers at 
control campuses reported a higher frequency of technology use for Professional Productivity 
{d - -0.17). Control teachers were somewhat more likely to report using technology for activities like 
keeping administrative records, communicating with students, and creating instructional materials. On 
the other hand, teachers at control campuses also were more Resistant to Integration (use of 
technology in their instruction) {d - -0.24). They were more likely to feel that computers are not a 
necessary part of instruction, using classroom computers is not a priority, and students’ use of 
computers is not practical. 
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Table 3.3. Group Differences for Teacher Technology-Related Variables 



Immersion 

Control 



Effect 

Variables 

Mean 

SD 

Mean 

SD 

f-value 

P 

Size 

Technology Proficiency'* 

4.47 

1.52 

4.58 

1.50 

-1.25 

.211 

-0.07 

Professional Productivity’’ 

2.92 

0.75 

3.05 

0.71 

-2.98 

.003* 

-0.17 

Students’ Technology Use’’ 

1.85 

0.74 

1.90 

0.72 

-1.21 

.227 

-0.07 

Integration** 

Technology Integration 

2.98 

1.48 

2.88 

1.41 

1.10 

.272 

0.06 

Learner-Centered Instruction 

3.64 

1.39 

3.65 

1.34 

-0.16 

.873 

-0.01 

Resistance to Integration 

2.15 

1.20 

2.45 

1.33 

-4.16 

.000* 

-0.24 

Collaboration’’ 

2.41 

0.74 

2.35 

0.74 

1.43 

.152 

0.08 


Notes. Immersion (n ranges from 588 to 603 teachers), Control (n ranges from 665 to 685 teachers), collahoration. 
*Statistically significant difference. Effect size is Cohen’s d. 

'“Items rated on a 7-point scale: 1 {not true of me now) to 7 {very true of me now). 

’’Items rated on a 5-point frequency scale: 1 {never) to 5 {almost daily). 


Student Characteristics 

It was also important to determine whether differences existed between sixth-grade students at 
immersion and control campuses, so student surveys measured a number of technology-related 
variables. Comparisons in Table 3.4 show that immersed and control students reported comparable 
levels of Technology Proficiency with a variety of technology applications. Both groups of students 
also reported similar Technology Use frequencies across school and core-content classes, and School 
Satisfaction was similar for both immersed and control students. 


Table 3.4. Group Differences for Student Technology-Related Variables 



Immersion 

Control 



Effect 

Variables 

Mean 

SD 

Mean 

SD 

f-value 

P 

Size 

Technology Proficiency 

2.81 

0.92 

2.85 

0.90 

1.73 

.084 

-0.04 

Technology Use in School 

2.05 

0.82 

2.05 

0.79 

-0.14 

.891 

0.00 

Use in EL A 

1.97 

0.84 

1.92 

0.78 

-1.21 

.228 

0.06 

Use in Math 

1.82 

0.85 

1.78 

0.80 

-0.73 

.468 

0.05 

Use in Science 

1.88 

0.85 

1.90 

0.82 

0.39 

.700 

-0.02 

Use in Social Studies 

1.80 

0.85 

1.95 

0.92 

2.72 

.007* 

-0.17 

Technical Problems 

2.18 

0.93 

2.24 

0.93 

2.21 

.027* 

-0.07 

Small-Group Work** 

2.72 

0.90 

2.84 

0.90 

4.50 

.000* 

-0.13 

School Satisfaction** 

3.74 

0.74 

3.73 

0.75 

-0.20 

.839 

0.01 

Self-Directed Learning 

4.63 

0.73 

4.55 

0.75 

-3.84 

.000* 

0.11 


Note. Immersion (n ranges from 825 to 2,186 sixth-grade students). Control (w ranges from 908 to 2,391 sixth- 


grade students). Scores range from 1 {completely false) to 7 {completely true) for the self-directed learning, from 
1 (7 can do this not at all or barely) to 5 (7 can do this extremely well) for technology proficiency, from 1 {never) 
to 5 {almost daily) for technology use, small group work, and technology use in subject-specific classes, from 1 
{never) to 5 {almost always) for technology problems, and from 1 {strongly disagree) to 5 {strongly agree) for 
student satisfaction with school. *Statistically significant difference. Effect size is Cohen’s d. 

Four statistically significant differences between groups were found; however, the mean differences 
and effect sizes were small. Students on immersed campuses reported higher levels of Self-Directed 
Learning (q? = 0.11), whereas students on control campuses more frequently reported Small-Group 
Work with other students {d - -0.13), having Technical Problems (-0.07), and Technology Use in 
Social Studies classes {d - -0.17). 
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In sum, the evidence from baseline data collected in fall 2004 suggests that the selection of control 
campuses by matching on key variables was effective in creating a comparable sample of comparison 
schools. Immersion and control schools are reasonably well matched on a number of important school, 
teacher, and student variables. Although a few statistically significant differences between groups have 
been identified, effect sizes for those differences are generally small, and significant differences are 
often due to large sample sizes. The collection of baseline data on an array of mediating and outcome 
variables from the study’s theoretical framework has provided a thorough understanding of existing 
conditions in schools. This information will be valuable in analyzing and interpreting data collected 
subsequently. Baseline data on variables will serve as statistical controls to adjust for pre-existing 
conditions that could confound results. 
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4. Technology Immersion — First-Year Implementation 

This section first describes the components of technology immersion and efforts to operationally 
define the treatment and ensure more consistent implementation across sites. Next, the measurement 
of implementation fidelity is explained. Finally, findings are presented on first-year implementation at 
the 22 treatment campuses. Information comes from both qualitative and quantitative data collected 
during the 2004-05 school year. 

Defining Technoiogy Immersion 

As a way to ensure consistent interpretation of technology immersion and comparability across sites, 
the Texas Education Agency issued a Request for Qualifications (RFQ) that allowed commercial 
vendors to apply to become providers of technology immersion packages (TEA, 2003). Although state 
statute gave a general description of technology immersion, the concept and its component parts 
needed to be defined operationally to foster consistency across sites. To that end, successful vendor 
applicants to the RFQ had to include the following six components in their plan: 

• A wireless mobile computing device for each educator and student on an immersed campus 
to ensure on-demand access to technology; 

• Productivity, communication, and presentation software for use as a learning tool; 

• Online instructional resources that support the state curriculum in English language arts, 
mathematics, science, and social studies; 

• Online assessment tools to diagnose students’ strengths and weaknesses or to assess their 
progress in mastery of the core curriculum; 

• Professional development for teachers to help them integrate technology into teaching, 
learning, and the curriculum; and 

• Initial and ongoing technical support for all parts of the package. 

Through a competitive application submission and expert review process, the TEA selected three lead 
vendors as providers of technology immersion packages (Dell Computer Inc., Apple Computer Inc., 
and Region 1 Education Service Center [ESC]). Table 4.1 provides an overview of the basic 
components within each package and the individual vendors that provide various components. 


Table 4.1. Technology Immersion Packages 



Apple 

N=b Schools 

Dell 

N=\5 Schools 

Region 1 ESC 
N=1 School 

Wireless laptop computer 

Apple 
iBook G4 

Dell Inspiron 
or Latitute 

Dell 

Inspiron 

Productivity software 

Apple- Works 

MS Office 
eChalk 

MS Office 
eChalk 

Online resources 

Various 

Various 

Various 

Online assessment 

AssessmentMaster 

i-Know 

i-Know 

Professional development 

Apple Model 

Co-nect 

Classroom Connect 

Technical and pedagogical 

Apple 

Dell 

ESC 1 

support 

Campus 

Campus 

Campus 
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Prices for technology immersion packages varied according to the numbers of students and teachers, 
the type of laptop computer, and the vendor provider. Package costs ranged from about $1,100 to 
$1,600 per student. Of the 22 immersion sites, 6 middle schools selected the Apple package, 15 
selected the Dell package, and 1 school selected the Region 1 ESC package (Dell computer) 

Wireless Laptops and Productivity Software 

All vendors offered a wireless laptop as the mobile computing device. Campuses could select either 
Apple laptops (iBook and MAC OSX) or Dell laptops (Inspiron or Latitude with Windows OS). For 
Apple laptops, AppleWorks provides a comprehensive suite of productivity tools, including Keynote 
presentation software, Internet Explorer, Apple Mail, iCal calendars, iChat instant messaging, and 
iLife Digital Media Suite (iMovie, iPhoto, iTunes, GarageBand, and iDVD). For Dell laptops, 
Microsoft Office includes Word, Excel, Outlook, PowerPoint, and Access. In addition, eChalk serves 
as a “portal” to other web-based applications and resources included in the immersion package and 
student-safe email solution. Region 1 ESC offered either Apple or Dell products. 

Online Instructional and Assessment Resources 

Immersion packages also include a variety of digital resources. Apple included the following online 
resources: netTrekker (an academic Internet search engine). Beyond Books from Apex Learning 
(reading, science, and social studies online), ClassTools Math from Apex Learning (complete math 
instruction), ExploreLearning Math and Science (supplemental math/science curriculum), KidBizSOOO 
from Achieve 3000 (differentiated reading instruction), and My Access Writing from Vantage Learning 
(support for writing proficiency). Dell, Inc. selected netTrekker (an academic Internet search engine) 
and Connected Tech from Classroom Connect (technology-based lessons and projects). Region I ESC 
selected Connected Tech but also added a variety of teaching and learning resources including 
Unitedstr earning (digital videos). Encyclopedia Britannica, EBSCO (databases), NewsBank, and K12 
Teaching and Learning Center. 

For the Apple package, AssessmentMaster (Renaissance Learning) provides a formative assessment in 
all four core subject areas. Both the Dell and Region 1 ESC packages provide i-Know (CTB McCraw 
Hill) for core-subject assessment. In addition, all campuses have access to the online Texas 
Mathematics Diagnostic System (TMDS) that is provided free of charge by the state. 

Professional Development 

Each immersion package includes a different professional development provider. Apple uses its own 
professional development model, whereas the Dell package relies on Co-nect (a commercial provider) 
to support professional development. Region 1 ESC uses a combination of service center support plus 
other services offered through Connected Coaching and Connected University. Although the 
professional development models and providers differ, they all include some common required 
elements, such as support for immersion package components, the design of technology-enhanced 
learning environments and experiences, lesson development in the core subject areas, sustained 
learning opportunities, and ongoing coaching and support. Individual districts and campuses must 
collaborate with vendors to develop their own professional development plans for teachers and other 
staff. 

Technical and Pedagogical Support 

Each technology immersion package provider also is required to provide campus-based technical 
support to advance the effective use of technology for teaching and learning. Apple designed a Master 
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Service and Support Program that leveraged its broad-based experience in one-to-one projects. Dell 
established a Call Center dedicated to technical support for TIP grantees as well as an 800 telephone 
number for hardware and software support. Region 1 ESC has an online and telephone HelpDesk to 
answer questions and provide assistance. 

In sum, the RFQ process allowed the creation of technology immersion packages with common 
elements. Although the treatment is not identical across sites, the specification of a comprehensive 
technology intervention with specific components represents a major accomplishment in Texas, a state 
that strongly endorses local control. Still, the complex nature of the treatment makes it critically 
important for researchers to document how and how well the immersion packages are implemented in 
order to explain any differences in outcomes between treatment and control schools that may emerge. 

Measuring Implementation Fidelity 

Program implementation involves a process of adapting a model program to local conditions and 
organizational dynamics. Thus, even well conceived programs typically progress through stages from 
initial struggles to get the program initiated to full implementation (e.g., Fullan, 1993; 1999). 
Consequently, in order to understand and interpret program effects, a well designed evaluation must 
assess progress toward the implementation of core program components undertaken to achieve 
intended goals and outcomes. For the technology immersion project, domains and related indicators 
were identified through a review of the technology immersion packages and interviews and 
discussions with TEA staff and immersion package providers. Implementation is measured as the 
fidelity with which elements of technology immersion attain the envisioned “ideal.” This approach 
involves gathering extensive data on various aspects of immersion at each of the treatment campuses 
and comparing that with the vision for “full” implementation. 

Researchers have developed indicators and scoring rubrics to measure the extent of implementation. 
Rating scales identify four levels of immersion: minimal (1), partial (2), substantial (3), and full (4). 
The overall level of Technology Immersion is a composite score derived from values for four 
domains: (a) Robust Access to Technology, (b) Technical and Pedagogical Support, (c) Professional 
Development, and (d) Resource Utilization. For each domain, a number of indicators are used to 
assess the quality of implementation. Scores are derived from various data sources including vendor 
records, interviews with principals and technology specialists, focus groups with teachers and 
students, surveys of teachers and students, and grant evaluation documents. 

A description of the implementation domains and indicators for technology immersion is provided in 
Table 4.2. Paragraphs to follow offer a general description of the measurement approach for the four 
domains. Appendix B provides additional technical detail on measuring implementation fidelity. 

Robust Access to Technology 

Indicators for Robust Access to Technology assess the nature and extent of student access to laptop 
computers as well as the frequency of student laptop use for core-content learning. Four indicators — 
School Access, Home Access, Application Use in Core Subjects, and Technology Frequency in Core 
Subjects — contribute to the domain score. Foremost, in an immersed school, students should have 
access to wireless laptops the entire school year. In the first implementation year of the project, 
however, student access to laptops varied substantially across schools due to variations in grant award 
dates, time requirements for planning and purchasing equipment, and school policies. Our measure of 
School Access was the number of days out of the 1 80-day school year that students actually had 
laptops available for use. Furthermore, on a fully immersed campus, students should have access to 
their wireless laptops for learning both in and outside of school. However, at some middle schools in 
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the first year, school policies, insurance issues, parent refusals to accept liahility, and penalties for 
student discipline and behavior problems limited student’ access to laptops. Thus, we also have 
gauged Home Access, the extent to which middle schools allowed students to have access to laptops 
outside of school (i.e., no access to complete access). 

Table 4.2. Description of Implementation Indicators for Technology Immersion 

Robust Access to Technology 

School Access: To what extent do students have access to wireless laptops throughout the school year. 

Home Access: To what extent do students have access to wireless laptops outside of the school. 

Application Use in Core Subjects: To what extent do students use particular technology applications in core- 
subject classrooms, such as a word processor for writing, a spreadsheet for calculation or graphing, Internet for 
research. 

Technology Frequency in Core Subjects: With what frequency do students use technology in core subject 
classes. 

Technical and Pedagogical Support 

Technical Support-Staffing: To what extent does district and campus staffing provide support for immersion. 
Technical Support-Problems: To what extent are technical problems barriers to technology immersion. 

Formal Coaching and Mentoring: To what extent do teachers receive coaching or mentoring from an internal 
source, such as another teacher or technology coordinator. 

Collegial Support: To what extent do teachers support each other in their efforts to integrate laptops into 
instruction and the curriculum. 

Professional Development 

Contact Hours: To what extent does the duration (hours) of technology-related professional development 
support the integration of technology into teaching, learning, and the curriculum. 

Span of Time: To what extent does ongoing professional development support the integration of technology. 

Collective Participation: To what extent do all teachers at immersed campuses participate in available 
professional development opportunities. 

Classroom Support: To what extent do teachers receive in-class modeling, coaching, or mentoring by 
professional development providers to support technology immersion. 

Resource Utilization 

Productivity Software: To what extent do students use productivity software (i.e., AppleWorks ov Microsoft 
Office) for learning activities and assignments. 

Curricular Resources: To what extent do teachers use curricular resources provided with technology 
immersion packages for lessons and assignments (e.g., NetTrekker, KidBiz, eChalk). 

Online Assessments: To what extent do core-subject teachers use online assessments to diagnose student 
strengths and needs. 

Note. See Appendix B for a full list of data sources used in the creation of these indicators. 

The potential for laptops to affect achievement depends largely on students’ opportunities to use 
technology for learning core academic content. Consequently, we have used items from student 
surveys to assess the extent to which students used particular technology applications in their core- 
subject classrooms (Application Use in Core Subjects). For example, survey items gauged how often 
students’ used a word processor to write a story or used software to learn and practice skills. As an 
additional indicator, we measured the consistency of students’ use of technology across the core 
subject areas (reading/English language arts, mathematics, science, and social studies classes). Student 
survey items measured the frequency with which teachers had students use technology in each of the 
subject areas (Technology Frequency in Core Subjects). 

Technical and Pedagogical Support 

The provision of Technical and Pedagogical Support for immersion by vendors and campus-level staff 
is a grant requirement. Accordingly, we measure four indicators of support: Technical Support- 
Staffing, Technical Support-Problems, Formal Coaching and Mentoring, and Collegial Support. 
Information for ratings comes from interviews with campus principals and technology leaders and 
through teacher survey items. Fevels of Technical Support-Staffing ranged from campuses with 
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“minimal technology support available” to campuses that had “full access to capable support staff with 
minimal or no waits for repair, assistance, or installation.” Accordingly, on immersed campuses, 
technical support and a healthy infrastructure are expected to alleviate technical problems that might 
interfere with the use of technology in the classroom. Measurements for the extent of Technical 
Support-Problems ranged from “pervasive and long waits” to “few problems and minimal waits.” 

In addition to technical support, each middle school was expected to have a dedicated staff person to 
provide ongoing pedagogical support. Thus, as a measure of Formal Coaching or Mentoring, we 
assessed the extent to which each teacher reported that he or she received “coaching or mentoring 
from an internal source, such as another teacher or technology coordinator.” Teachers in an immersed 
school also are expected to support each other in their efforts to integrate laptops into instruction and 
the curriculum. Therefore, items on teacher surveys provided a measure of Collegial Support (i.e., the 
frequency of teachers’ interactions with their colleagues on teaching and learning with technology). 

Professional Development 

In constructing measures of professional development, we draw from research conducted on the 
effectiveness of the Eisenhower Professional Development Program (e.g.. Caret, Porter, Desimone, 
Birman, & Yoon, 2001). Key features of quality professional development provide a framework for 
examining dimensions of vendors’ professional development models. We have adapted the dimensions 
to align with our project objectives and approaches. First, we measured the total number of Contact 
Hours teachers spent in professional development, and we also determined the period of time in days, 
weeks, or months over which the activity was spread (Time Span). In addition, we assessed the extent 
to which all teachers on immersed campuses participated in available professional development 
opportunities. Based on teacher survey data, we estimated Collective Participation (the percentage of 
teachers at each campus participating in 17 or more hours of professional development). Professional 
development models for technology immersion also were required to include a classroom support 
component, so we measured Classroom Support as the percentage of teachers involved in modeling, 
coaching or mentoring. Data for professional development measures come from online professional 
development logs submitted by districts to the TEA during the 2004-05 school year, documents 
provided by vendors, and professional development items included on teacher surveys. 

Resource Utilization 

The technology immersion packages include a variety of instructional and assessment resources 
designed to extend, supplement, or enhance core-subject learning. Relying on multiple data sources, 
we assessed the extent to which teachers and students utilized Productivity Software, Instructional 
Resources, and Online Assessments provided with each vendor package. Information on resource 
availability and use comes from focus group discussions with samples of students and teachers at each 
campus, observations in core-subject classrooms, and items on the teacher survey. 

First, wireless laptops at immersed campuses were loaded with productivity software (i.e., either 
Appleworks or Microsoft Office) for students to use as a learning tool. Consequently, we used items 
from the teacher survey to gauge how often teachers had students use Productivity Software for 
learning activities in their typical classroom. Since teachers also had access to a variety of digital 
resources, we also gathered information during focus groups and classroom observations to determine 
how often (i.e., never or rarely to almost daily) that teachers used Curricular Resources for lessons or 
assignments. In addition, immersion packages provided teachers with online formative assessments, so 
we assessed the pervasiveness of core-subject area teachers’ use of Online Assessments to diagnose 
students’ academic strengths and needs. 
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Progress Toward Immersion 


Findings in Figure 4.1 and Table 4.3 illustrate the progress of the 22 middle schools toward 
technology immersion in the first year. Growth is measured on a 4-point scale {minimal, partial, 
substantial, and full immersion) across the four domains (Robust Access to Technology, Technical 
and Pedagogical Support, Professional Development, and Resource Utilization). Results suggest that 
campuses have made strides toward immersion in the first year, but the level of implementation varies 
by campus and no middle school has achieved/u/Z immersion. Average scores across immersion 
domains indicate that the majority of campuses (20) achieved only partial immersion in the first year, 
while 2 campuses reached substantial immersion. Findings for each technology immersion domain are 
presented below. 



Robust Access Technical & Pedagogical Professional Development Resource Utilization 

Support 


Technology Immersion Domains 


□ Minimal D Partial □ Substantial DFulTI 


Figure 4.1. First-year progress toward technoiogy immersion by domain scores. 

Robust Access to Technoiogy 

School Access. Due to logistical procedures in the awarding of two rounds of grant funds, most of the 
campuses had a late start. Teachers for the most part did not receive their laptops until late September 
through early December 2004. Student laptop “rollout” was delayed even more. About half of the 22 
campuses distributed laptops to students during the first semester of the school year, with rollouts 
occurring during October, November, and December 2004. The remainder of campuses distributed 
laptops to students in January and February 2005. Due to delays in laptop rollout, the amount of time 
students had their laptops varied by campus, with many students having less than four months of 
laptop use. Student days with laptops ranged by campus from 72 to 144 days (out of a 180-day school 
year), with a mean of 105 days. Accordingly, School Access at campuses was rated as minimal (8 
campuses), partial (9 campuses), or substantial (5 campuses). 
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Table 4.3. First-Year Implementation of Technology Immersion 


Middle 

School 

(MS) 

Robust 

Access 

Technical & 
Pedagogical 
Support 

Professional 

Development 

Resource 

Utilization 

Overall 

Level of Immersion 

MS 1 

3.00 

3.50 

3.00 

1.67 

2.79 

Substantial 

MS 2 

3.00 

2.00 

3.75 

1.67 

2.60 

Substantial 

MS 3 

2.50 

3.25 

2.75 

1.67 

2.54 

Partial 

MS 4 

2.50 

3.25 

2.75 

1.67 

2.54 

Partial 

MSS 

2.25 

3.25 

2.50 

2.00 

2.50 

Partial 

MS 6 

2.25 

3.00 

3.25 

1.33 

2.46 

Partial 

MS 7 

3.00 

2.25 

2.75 

1.67 

2.42 

Partial 

MS 8 

2.00 

2.75 

2.75 

2.00 

2.38 

Partial 

MS 9 

2.75 

2.50 

2.75 

1.33 

2.33 

Partial 

MS 10 

2.25 

2.50 

3.25 

1.33 

2.33 

Partial 

MS 11 

2.75 

2.00 

2.75 

1.33 

2.21 

Partial 

MS 12 

2.75 

2.50 

2.25 

1.33 

2.21 

Partial 

MS 13 

2.50 

2.00 

2.25 

2.00 

2.19 

Partial 

MS 14 

2.25 

2.25 

2.75 

1.33 

2.15 

Partial 

MS 15 

2.75 

2.25 

2.25 

1.33 

2.15 

Partial 

MS 16 

2.00 

2.75 

2.50 

1.33 

2.15 

Partial 

MS 17 

2.75 

2.00 

2.25 

1.33 

2.08 

Partial 

MS 18 

2.50 

2.50 

1.25 

2.00 

2.06 

Partial 

MS 19 

2.00 

2.75 

1.50 

1.67 

1.98 

Partial 

MS 20 

2.50 

2.25 

1.50 

1.67 

1.98 

Partial 

MS 21 

1.75 

2.50 

2.25 

1.33 

1.96 

Partial 

MS 22 

1.75 

2.50 

1.75 

1.33 

1.83 

Partial 


Note. Mean campus scores based on the 4-point immersion scale: minimal (1), partial (2), substantial (3), and 
full (4). In assigning the level of immersion, scores at or above 1.60, 2.60, or 3.60 are rounded to the next level. 


Home Access. Students’ access to laptops outside of school also varied. While students at 14 
campuses could take laptops home, at 6 campuses students were not allowed to use laptops outside of 
the school, and student laptop use at 2 campuses was restricted to special assignments. Even when 
students were allowed to take laptops home, access to laptops for some students was reduced hy the 
confiscation of laptops for disciplinary reasons, parents’ inability to pay insurance or other fees, or in a 
few cases, parents’ refusals to let students have laptops. In view of varying circumstances. Home 
Access was rated as minimal (6 campuses), partial (2 campuses), or substantial (14 campuses) rather 
than full immersion. 

Application use in Core Subjects. Another indicator of robust technology access is a students’ use of 
technology applications for learning activities in core-subject courses. In the first year, students at 
some middle schools had more frequent opportunities to use their laptops in core subjects than other 
students. Based on the campus average for students’ mean application use survey scores, the level of 
Application Use in Core Subjects was rated as either partial (6 campuses, students use applications 
rarely, e.g., a few times a year) or substantial (16 campuses, students use applications sometimes, e.g., 
once or twice a month). No campuses received either a minimal rating (students use applications never 
or almost never) or a fully immersed rating (students use applications often, e.g., once or twice a week 
or almost daily). 

Technology Frequency in Core Subjects. An additional indicator of students’ technology use was 
their rating of the frequency with which teachers have them use technology in each of their four core 
classes. The four ratings were combined for each student and averaged across the campus to create a 
mean technology use frequency score. Using campus averages. Technology Frequency in Core 


39 




Subjects was rated as partial (6 campuses), substantial (14 campuses), ox fully immersed (2 
campuses). 

Mean campus rating. The campus rating for Robust Access to Technology was calculated as the 
average of scores for School Access, Home Access, Application Use in Core Subjects, and 
Technology Frequency in Core Subjects. Campus mean implementation scores ranged from 1.75 to 
3.00 on the 4-point immersion scale, with 14 campuses rated as partial immersion and 8 campuses as 
substantial immersion. 

Technical and Pedagogical Support 

A key aspect of the immersion project is dedicated support for the implementation of technology 
immersion. Although the type and extent of support varies by campus, extensive grant and local 
resources have been invested in providing assistance to teachers and students in using technology. 

Even so, first- year results show that the level of support differs by campus. Indicators for Technical 
and Pedagogical Support included Technical Support-Staffing, Technical Support-Problems, Formal 
Coaching or Mentoring, and Collegial Support. 

Technical Support-Staffing. Through qualitative analyses of data from interviews and focus groups, 
each campus was assigned one of four levels of staffing. Nine campuses had minimal technical support 
available (little or no access to technology support staff capable of completing repair, assistance, or 
installation). For these campuses, no mechanism for technical support had been established and no 
trained or experienced person provided technical support. An additional six campuses had partial 
technical support available (limited access to capable support staff and long waits for repair, 
assistance, or installation). Technical support providers on these campuses were either shared among 
several campuses, were shared among several duties (such as teaching or, in one case, serving as a 
principal), or were contracted local companies. The net result was access to inconsistent and part-time 
support. On five campuses, access to one key, full-time technology specialist amounted to substantial 
access (almost full access to capable support staff with moderate waits). On these campuses, 
technology specialists often felt overwhelmed with technical support tasks, such as troubleshooting, or 
were required to split their duties between technical and instructional support, often resulting in an 
overemphasis on the former. Only two campuses appeared to he fully immersed (full access to capable 
support staff with minimal or no waits for repair, assistance, or installation). On these campuses, full- 
time, experienced and dedicated staff was in place to provide technical support. In these cases, a 
second person(s) was hired to provide instructional support. 

Technical Support-Problems. Since some campuses had fully capable technical support staff who 
could alleviate problems while other campuses had little or limited access to capable support, it is not 
surprising that some surveyed teachers reported substantial problems with equipment, Internet access, 
and waits for repairs, whereas on other campuses, teachers reported fewer problems. Teachers’ mean 
ratings of the extent of Technical Problems (i.e., working condition of computers, speed and 
dependability of Internet connections, timely assistance) were averaged to create a mean campus 
score. Mean ratings indicate that four campuses achieved partial immersion (substantial technical 
problems and long waits) and 17 campuses reached substantial immersion (some problems and 
moderate waits). In contrast, one campus was regarded as fully immersed (few problems and minimal 
waits). 

Formal Coaching or Mentoring. Even though each middle school was expected to have a dedicated 
staff person on campus to provide ongoing pedagogical support, teachers’ reports on the availability of 
coaching or mentoring from an internal source, such as another teacher or technology coordinator, 
varied substantially by campus. On one campus, immersion was considered minimal because 50% or 
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more of core-content teachers reported that they rarely or never received classroom coaching or 
mentoring, and on five other campuses, the level of immersion was rated as partial because 60% or 
more of teachers received classroom coaching or mentoring only sometimes. In contrast, at 10 
campuses the immersion level was rated as substantial because 60 to 79% of teachers reported that 
they frequently received classroom coaching or mentoring. Six campuses received fully immersed, 
ratings, with 80% or more of core-content teachers indicating that they frequently received classroom 
coaching or mentoring from an internal source. 

Collegial Support. Teachers at immersed campuses also look to their peers for support as they 
integrate wireless laptops and new educational resources into their instruction and the existing 
curriculum. Thus, for each campus, we constructed a mean measure of teachers’ self-reported 
interactions with their colleagues regarding technology (e.g., informal discussions on integration, 
exchanging feedback on student work, working with a subject-area peer to develop a lesson). The 
mean level of Collegial Support calculated for each campus showed that middle school teachers 
expressed differing levels of support from colleagues. At one campus, the level of Collegial Support 
was rated as minimal (teachers never or almost never interact with colleagues on technology 
integration), whereas the extent of support was rated as partial on 10 campuses (teachers rarely, e.g., a 
few times a year, interact with colleagues on technology integration). On the remaining 1 1 campuses, 
the level of Collegial Support was rated as substantial (teachers sometimes, e.g., once or twice a 
month, interact with colleagues). No campus was considered to he fully immersed, with teachers often 
or almost daily interacting with colleagues on technology immersion. 

Mean campus rating. The campus rating for Technical and Pedagogical Support was calculated as 
the average of scores for Technical Support — Staffing, Technical Support — Problems, Formal 
Mentoring and Coaching, and Collegial Support. Campus mean implementation scores ranged from 
2.00 to 3.50 on the 4-point immersion scale, with 14 campuses rated as partial immersion and 8 
campuses as substantial immersion. No campuses were considered as minimally or fully immersed. 

Professional Development 

All districts and campuses provided professional development for teachers and staff during the first 
year. However, the extent and type of professional development varied according to the particular 
vendor that provided training, the number of days allocated by each campus for professional 
development, and the extent of teacher participation in available opportunities. Our analyses of 
Professional Development encompassed indicators for Contact Hours, Time Span, Collective 
Participation, and Classroom Support. 

Contact Hours. Teachers at campuses typically indicated participation in four or less days of 
technology-related professional development during the school year. Based on mean survey responses, 
teachers at five campuses averaged 8 to 16 professional development hours (partial immersion), 
teachers at 14 campuses had 17 to 32 professional development hours (substantial immersion), and 
teachers at three campuses had 32 or more hours of technology-related professional development, on 
average, during the year (full immersion ). 

Time Span. For some schools, professional development was spread over a brief period of time, 
whereas at other campuses, professional development encompassed the entire school year. Vendor and 
campus reports on professional development activities suggest that the implementation level was 
minimal at four campuses (professional development spanned one month or less), partial at three 
campuses (professional development spanned less than four months), and substantial at 10 campuses 
(professional development spanned four to eight months). Five campuses reached /«// immersion, with 
professional development spanning nine or more months. 
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Collective Participation. In the first year, technology-related professional development typically 
involved some teachers hut not all or even almost all. As a measure of Collective Participation, we 
used teacher survey data to determine the percentage of teachers at each school who participated in 17 
or more hours of technology-related professional development. Based on campus percentages, the 
level of immersion for collective participation was minimal for eight campuses (less than 60% of 
teachers participate in 17 or more hours), partial for seven campuses (60 to 79% of teachers 
participate in 17 or more hours), substantial for two campuses (80 to 89% of teachers participate in 17 
or more hours, wd fully immersed for five campuses (90% or more of teachers participate in 17 or 
more hours of technology -related professional development). 

Classroom Support. Classroom Support is a measure of the extent to which core-content teachers 
received coaching or mentoring from an external (non-school) source as stipulated in the technology 
immersion professional development models. Teachers’ survey ratings of the frequency of Classroom 
Support suggest that the level of implementation was minimal at eight campuses (50% or more of 
teachers rarely or never receive coaching or mentoring ), partial at three campuses (60% or more of 
teachers sometimes receive coaching or mentoring), and substantial at 1 1 campuses (60% to 79% of 
teachers frequently receive coaching or mentoring). No campus achieved^// immersion, with 80% or 
more of core-content teachers reporting frequent classroom support. 

Mean campus rating. The campus rating for Professional Development was calculated as the average 
of scores for Contact Hours, Time Span, Collective Participation, and Classroom Support. Campus 
mean implementation scores ranged from 1.50 to 3.75 on the 4-point immersion scale, with three 
campuses rated as minimal immersion, eight campuses as partial immersion, and 10 campuses as 
substantial immersion. One campus was considered to he fully immersed. 

Resource Utilization 

Technology immersion packages incorporated a variety of instructional and assessment resources 
designed to augment core-content teaching and learning. In view of that, we assessed the extent to 
which teachers and students utilized immersion resources, including indicators for Productivity 
Software, Curricular Resources, and Online Assessments. 

Productivity Software. Wireless laptops were loaded with productivity software {either AppleWorks 
or Microsoft Office) for use as a learning tool. Teachers’ responses to four survey items gauged the 
extent to which they had their students use productivity software components in a typical class for 
activities such as writing, calculations and graphing, database development, or presentations. Mean 
campus ratings for teachers indicated that the level of Productivity Software implementation was 
partial at 13 campuses (students rarely, e.g., a few times a year, use productivity software) and 
substantial at nine campuses (students sometimes, e.g., once or twice a month, use productivity 
software). 

Curricular Resources. Technology immersion packages included a variety of informational resources 
(e.g., NetTrekker, Beyond Books, EBSCO) and subject-specific programs (e.g.. Connected Tech, 

KidBiz 3000, My Access Writing, and Explore Learning). Through a qualitative analysis of 
information gathered during focus groups with teachers and students and classroom observations, we 
assigned each campus one of four levels of implementation for Curricular Resources. At 16 middle 
schools, the implementation level for Curricular Resources was rated as minimal immersion (resources 
are never or rarely used for lessons or assignments), and six schools achieved only partial immersion 
(resources are sometimes used for lessons or assignments). Teachers’ comments revealed that many 
were frustrated by simultaneously learning how to use the new wireless technology and learning how 


42 



to infuse new resources into the existing curriculum. Teachers mentioned a need for more time, 
training, and guidance in how to use the resources. Even though teachers did not use the immersion 
package resources extensively, many reported that they frequently accessed instructional resources 
from the Internet, used technology-hased materials from their adopted textbooks, or used other 
programs that their school had purchased previously. 

Online Assessments. Teachers also had online formative assessments for the core subjects as part of 
their immersion packages (either i-Know or Assessment Master), and mathematics teachers could use 
the state’s online Texas Mathematics Diagnostic System (TMDS). The qualitative analysis of teachers’ 
responses during focus groups provided the basis for assigning the implementation level for Online 
Assessments. Similar to curricular resources, very few teachers used the online assessments, and those 
who used them, did so infrequently. Again, teachers cited a lack of time, insufficient training, and 
technical problems in using the assessments. Thus, the level of immersion for Online Assessments was 
minimal at each of the 22 middle schools (very few teachers used online assessments to diagnose 
student strengths and needs). 

Mean campus rating. The campus rating for Resource Utilization was calculated as the average of 
scores for Productivity Software, Curricular Resources, and Online Assessments. Campus mean 
implementation scores ranged from 1.33 to 2.00 on the 4-point immersion scale, with 11 campuses 
rated as minimal immersion and 1 1 campuses rated as partial immersion. Although most teachers were 
introduced to the resources during the initial year and took first steps toward using at least some of 
them, overall use was sporadic. 


Implementation Challenges 

Research studies and practical wisdom reveal that change takes time and is a process of growth (e.g., 
Senge, 1999). Understanding the process and challenges that limit progress, however, helps to nurture 
continued progress. Evidence gathered from work in the field on the challenges encountered during the 
initial year revealed several factors that impeded progress toward immersion: 

• Insufndent planning. Since many districts and campuses had a late start (i.e., REA process, 
funding), planning and project initiation often occurred “on the fly.” Understandably, this 
undermined implementation. 

• Outdated infrastructure. Outdated school infrastructure and technical issues have impeded 
progress on many campuses. Districts have invested substantial local resources on 
unanticipated costs such as building the infrastructure to support wireless technology. 

• Teacher readiness and receptivity. Teacher attitude, understanding of immersion, 
technology knowledge and skills, classroom management, and propensity toward technology 
use also varied substantially by classroom and school. 

• Inconsistent policies and practices. Policies and practices related to laptop access and use 
evolved during the year and varied by district (e.g., Internet access at home, access to games 
and email, disciplinary infractions). 

• Inconsistent professional development. The extent and type of professional development 
varied across schools and for individual teachers within schools. 

• Pressure to improve TAKS scores. Pressure to improve scores on the state assessment 
(TAKS) impeded change as teachers often attempted to maintain the status quo rather than 
try new and untested methods and materials. 
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• Leadership. The extent and continuity of leadership varied hy district and campus (see 
description helow). 

Although not addressed specifically in the immersion package, leadership emerged as a critical factor 
contributing to and explaining the level of implementation. Factors that appeared to affect 
implementation fidelity included leadership instability, the level of leadership commitment to the 
project, the creation of a leadership team, and the level of vendor support. First, leadership instability 
was an issue because at some campuses principals were on leave for several months, while on other 
campuses principals resigned and were replaced by interim principals. In other instances, principals 
announced that they would be leaving at the end of the school year, so teachers anticipated a 
leadership change. Additionally, in several districts, superintendents resigned and were replaced by 
interim superintendents or new superintendents who were not involved in the grant application 
process. 

Administrative commitment to full implementation of technology immersion also varied. In some 
districts, this project was a high priority and received strong financial and policy support at all levels 
of the system (superintendent, district leaders, and campus leaders). For example, in one small district, 
the superintendent recognized that technology was central to their students’ future, so he was directly 
involved in planning and visited the middle school on a regular basis. This “hands-on” administrative 
involvement communicated expectations for success to campus leaders, teachers, and students. In 
larger districts, superintendents were less likely to be directly involved, but in some cases, a well 
connected central administrator played a key role in establishing expectations for the project. 

Districts and campuses that appeared to be making headway in immersing middle schools in 
technology had often created leadership teams with key roles and responsibilities. Effective teams 
typically included at least one central administrator, a principal, and a campus specialist (technology 
and/or instructional). Although teachers were frequently consulted on project activities, they were less 
likely to be part of the formal decision-making team. Leadership from vendors was also important. In 
some cases, educators cited vendors’ key roles in planning, decision making, and problem solving. 

In sum, the low level of implementation of technology immersion components is an important 
consideration in the interpretation of first-year effects. Findings on campuses’ progress across various 
immersion components suggest that, at the end of the first year, overall campus implementation was 
most frequently only partial rather than substantial ox full immersion. 
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5. Effects of Technology Immersion on Schools and Teachers 

In the theoretical framework, researchers posited that given quality implementation of immersion (i.e., 
robust wireless technology access, effective use of online curricular and assessment resources, 
professional development supporting curricular integration, and adequate technical and pedagogical 
support to maintain an immersed campus), one might expect school-level improvements in measures 
of classroom technology integration, innovative culture, technical support, and parent and community 
support. Effective leadership and system support should improve implementation quality. 

This improved school environment should in turn lead to teachers who have greater technology 
proficiency, use technology more for their own professional productivity, and collaborate more with 
their peers. Moreover, teachers in immersed schools will have students use technology more in their 
classrooms, and they will use laptops as a tool to increase the intellectual challenge of lessons. 

Findings on the effects of technology immersion on various school- and teacher-mediating variables 
come from an online survey of teachers completed in fall 2004 (September to October) and again in 
spring of 2005 (April to May). Teachers responded to items pertaining to their perceptions of school 
technology and their own personal and classroom experiences with technology. In fall, 1,271 teachers 
completed the survey (97% of grades 6-8 teachers, 97% of immersed teachers, and 98% of control 
teachers). In spring, 1,144 teachers (88% of all teachers, 87% of immersed teachers, and 88% of 
control teachers) completed the survey. Additionally, researchers conducted classroom observations 
in a sample of sixth-grade core-subject classrooms in fall (64 treatment teachers and 64 control) and 
again in spring (124 treatment teachers and 123 control) to gather information on instructional 
practices and changes across time. 

School and Teacher Mediating Variabies — HLM Anaiysis 

On one part of surveys, teachers responded to items pertaining to their perceptions of school-level 
technology variables. Teachers were asked to rate the strength of their agreement with statements on a 
5-point scale ranging from 1 {strongly disagree) to 5 {strongly agree). Five distinct factors emerged 
from a factor analysis: Feadership and System Support (12 items). Classroom Technology Integration 
(4 items). Technical Support (5 items). Innovative Culture (4 items), and Parent and Community 
Support (2 items). Cronbach’s alpha coefficients for the school-level scale scores ranged from 
acceptable (0.67) to excellent (0.91). 

The teacher surveys also collected data on teacher mediating variables. First, for Technology 
Proficiency, teachers rated their skills in using various technology applications on a 7-point scale: 1 
and 2 {not true of me now)\ 3, 4, and 5 {somewhat true of me now)', and 6 and 7 {very true of me now). 
Teachers also rated the frequency with which they use technology for Professional Productivity 
(administrative and instructional tasks) on a 5-point scale: 1 {never), 2 {rarely: a few times a year), 3 
{sometimes: once or twice a month), 4 {often: once or twice a week), and 5 {almost daily). Fikewise, 
teachers rated the frequency of their Students’ Technology Use and Collaboration (with colleagues) on 
the 5-point frequency scale. 

In addition, teachers responded to items measuring technology integration on a 7-point scale, ranging 
from 1 {not true of me now) to 7 {very true of me now). Confirmatory factor analysis of these items, 
which were adapted from the Fevels of Technology Implementation (FoTi) Questionnaire (Moersch, 
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2001), showed reasonable fit indices for a model having Technology Integration, Learner-Centered 
Instruction, and Resistance to Integration as latent variables. Cronbach’s alpha coefficients for the 
teacher-level scale scores ranged from 0.70 (the 3-item Resistance to Integration scale) to 0.97 (the 27- 
item Technology Proficiency scale). (See technical appendix for details.) 

The analyses that follow contrast immersion and control teachers before and after one school year of 
the implementation of the Technology Immersion Pilot project. Immersion effects are estimated for 
each of the scales described above. We analyzed the effects of immersion on teachers’ perceptions of 
technology and self-reported proficiencies using a two-level hierarchical linear model (HLM).^ 

Teacher-Level Model 

In the teacher-level model, spring 2005 survey scale scores were regressed on fall 2004 scale scores, 
teaching experience in years, technology certification status® (0 if not certified, 1 if certified), and 
gender (0 if male, 1 if female). That is, 

Yij - Poj + f^ij(Fall 2004 scale score) + /) 2 j(Teaching experience) + /)3j(Teacher certification) 

+ P4j( Gender) + pj. 


With all 2005 survey scales, significant variation was found across middle schools (a justification for 
using HLM). The intraclass correlations ranged from 0.028 to 0.244 (see Table 5.5). Thus, the school 
means (fioj) were specified as randomly varying. The coefficients for fall 2004 scale score (J) ij) were 
specified as fixed unless the reduction in the deviance statistic (significant chi square) with the more 
complex model justified a random specification. The coefficients for experience, gender, and 
certification status were specified as fixed. 

School-Level Model 

A school-level model was developed to answer the question of whether immersed schools had higher 
scale scores than control schools, after controlling for initial scale scores and experience, gender, and 
certification status. That is, 

foj - yoo + yoi(Immersion dummy) + poj- 


The immersion dummy was an indicator variable with a value of 1 for an immersed school and a value 
of 0 for a control school. 


Effects of Immersion on Schools 

Data from analyses of school variables are summarized in Table 5.1. Additional statistical details for 
the HLM models are reported in Tables 5.2 and 5.3 for school-level scales. Summary results in 
Table 5.1 show that technology immersion had a statistically significant effect on teachers’ 
perceptions of four of the five school-level technology variables. For three scales, the size of the effect 


^ HLM was used for data analysis because teachers are clustered within middle schools. As a result, because of selection 
processes (e.g., schools may attract similar types of teachers) and shared common backgrounds, teachers within schools may 
be more similar to each other than are teachers from different schools. Consequently, measures within schools may not be 
independent, and may be more highly correlated than measures of teachers from different schools. (Note that when a 
clustering effect is absent, there is no need to utilize HLM.) Ignoring clustering results in aggregation bias and mostly 
underestimated standard errors. However, HLM makes no assumption about independence and it estimates the degree of 
clustering of measures and uses this estimate in the calculation of the precision with which immersion effects are estimated 
(Raudenbush & Bryk, 2002). 

^ Texas technology applications certificates include Technology Applications 8-12 and EC-12, Computer Science 8-12, and 
Master Technology Teacher. 
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(extent to which the phenomenon exists) was moderate to large. After controlling for fall 2004 scale 
scores, teacher experience, technology certification status, and gender, there were no significant 
differences in the spring 2005 scores of teachers in immersed schools and teachers in control schools 
on the Technical Support scale. Even so, teachers in treatment schools were more likely to report that 
“most of our school computers are kept in good working condition” and “my requests for technical 
assistance are addressed in a timely manner.” 


Table 5.1. Immersion Effects on School Technology 


School-Level Scales 

Immersion Effect Net of 
Fall Score, Teacher 
Experience, Gender, 

& Certfication Status 

Magnitude of 
Effect (d) in Standard 
Deviation Units“ 

Leadership and System Support 

Yes 

0.20 (small) 

Classroom Technology Integration 

Yes 

0.56 (large) 

Technical Support 

No 

0.19 (small) 

Innovative Culture 

Yes 

0.35 (moderate) 

Parent and Community Support 

Yes 

0.49 (moderate) 


Source: Online teacher surveys conducted in fall 2004 and spring 2005 

Note. Effect size is Cohen’s d. The interpretation is that anything greater than 0.5 is large, 0.5-0. 3 is 
moderate, 0.3-0. 1 is small, and anything smaller than 0.1 is trivial. 


For other school-level variables, after controlling for fall 2004 school-level scale scores, teacher 
experience, certification status, and gender, teachers in immersed schools had higher scale scores than 
teachers in control schools on: 

• Leadership and System Support (0.12 points, 0.20 standard deviation units), 

• Classroom Technology Integration (0.44 points, 0.56 standard deviation units), 

• Innovative Culture (0.23 points, 0.35 standard deviation units), and 

• Parent and Community Support (0.37 points, 0.49 standard deviation units). 

Teachers’ responses to individual scale items help to explain the positive effects of immersion on 
school mediating variables. 

Leadership and System Support 

Control teachers’ ratings of their schools’ support for technology remained virtually the same from fall 
to spring. In contrast, teachers in immersed schools rated administrative leadership and support higher 
on each of the scale items. The greatest differences between teachers centered on expectations, 
planning, leadership, and support for technology. Treatment teachers were far more likely than control 
teachers to believe: 

• In this school, there are clear expectations that technology will be used to enhance student 
learning (M = 4.2 on the 5-point scale, -1-0.5 points), 

• Teachers receive adequate administrative support to integrate technology into classroom 
practice (M = 4.0, -1-0.4 points), 

• The principal is an effective leader for instructional technology 
(M = 3.8, -1-0.4 points), and 

• Our school has a well-developed technology plan that guides all technology integration 
efforts (M = 3.7, -1-0.4 points) 
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Table 5.2. Immersion (Fixed) Effect Analyses of School-Level Scales 


School-Level 

School-Level Scale Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Leadership and System Supporf 

Base 

3.405 

0.112 

30.28*** 

Fall score 

0.665 

0.039 

17 01*** 

Experience (pooled) 

0.002 

0.001 

1.34 

Certification status 

0.213 

0.102 

2.09* 

Gender 

0.042 

0.020 

1.63 

Immersion dummy 

0.138 

0.051 

2.70* 

Classroom Technology Integration^ 

Base 

3.150 

0.156 

20.25*** 

Fall score (pooled) 

0.528 

0.032 

16.66*** 

Experience (pooled) 

-0.004 

0.002 

-1.95 

Certification status 

0.163 

0.145 

1.12 

Gender 

0.112 

0.044 

2.58* 

Immersion dummy 

0.444 

0.065 

6.78*** 

Technical Supporf 

Base 

3.037 

0.192 

j5 79*** 

Fall score 

0.467 

0.034 

13.57*** 

Experience (pooled) 

0.002 

0.002 

1.11 

Certification status 

0.397 

0.187 

2.13* 

Gender 

0.025 

0.054 

0.46 

Immersion dummy 

0.145 

0.079 

1.85 

Innovative Culture^ 

Base 

3.660 

0.150 

24.02*** 

Fall score 

0.493 

0.039 

12.48*** 

Experience (pooled) 

0.005 

0.001 

3.64* 

Certification status 

0.081 

0.141 

0.57 

Gender 

0.113 

0.041 

2.73** 

Immersion dummy 

0.225 

0.049 

4.56*** 

Parent, Community Supporf 

Base 

3.017 

0.158 

j9 j2*** 

Fall score (pooled) 

0.435 

0.038 

11.57*** 

Experience (pooled) 

0.002 

0.002 

0.86 

Certification status 

0.263 

0.155 

1.69 

Gender 

0.138 

0.043 

3.22** 

Immersion dummy 

0.374 

0.050 

7 5j*** 


*p < .05; **p < .01; ***p < .001. 
“Items rated on a 5-point scale. 
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Classroom Technology Integration 

This scale reflects the extent to which the school provides adequate classroom technology resources 
and sufficient training to use technology for instruction, and accordingly, the extent to which teachers’ 
use technology to address student technology standards, assess student performance, and plan 
instruction. As expected, immersed and control teachers’ responses differed most on items related to 
technology resources: 

• Students have adequate access to technology resources in my classroom 
{M - 3.6, +1.0 points), and 

• I incorporate the TEKS for student technology applications into my content area (M =4.1, 
+0.4 points). 

Surprisingly, considering the professional development component of immersion, there was only a 
small difference between teachers in their belief that, “I have received sufficient training to incorporate 
technology into my instruction” (M = 3.7 for treatment teachers and M - 3.5 for control). Responses 
may reflect immersion teachers’ increasing awareness of the challenges encountered in integrating 
classroom technology. 

Innovative Culture 

This scale intended to capture teachers’ shared understanding of technology, receptivity to technology, 
as well as their propensity for innovative practices. Although teachers in immersed schools rated each 
of the items higher than control teachers, the largest differences suggested that treatment teachers were 
more likely to: 

• share an understanding about how technology will be used to enhance learning 
(M = 3.9, +0.4 points) and 

• be unafraid to learn about new technologies and use them with their classes 
(M = 3.9, +0.3 points). 

Parent and Community Support 

The higher scores for teachers in immersed schools on items related to parent support for the school’ s 
emphasis on technology (M =3.8, +0.5 points) and active support from the community for 
instructional efforts with technology (M = 3.7, +0.3 points) appear to be reflect the strength of parent 
and community connections built through the implementation of one-to-one technology. All parents 
had to work with the schools regarding laptops. In addition, parents and community members often 
heard about technology immersion through the media or by participating in campus events. 

Effects of Immersion on Teachers 

For teacher-level measures, results were universally positive (see Tables 5.3 and 5.4). Technology 
immersion had a statistically significant effect on teachers’ self-reported ratings of six teacher-level 
technology variables. Effect sizes were typically moderate to large, with the greatest impact of 
immersion on teachers’ integration of technology in their classes and their students’ use of technology 
in the classroom. For one variable. Resistance to Integration, teachers in immersed schools reported 
lower levels of resistance than teachers in control schools, but differences were statistically 
insignificant. 
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After controlling for fall 2004 teacher-level scale scores and teacher experience, technology 
certification status, and gender, teachers in immersed schools scored higher than teachers in control 
schools on: 

• Technology Proficiency (0.22 points, 0.16 standard deviation units), 

• Professional Productivity (0.27 points, 0.37 standard deviation units), 

• Students’ Technology Use (0.57 points, 0.70 standard deviation units), 

• Collahoration (0.32 points, 0.41 standard deviation units), 

• Technology Integration (1.12 points, 0.73 standard deviation units), and 

• Learner-Centered Instruction (0.41 points, 0.30 standard deviation units). 


Table 5.3. Immersion Effects on Teacher Variables 


Scale and Type 

Immersion Effect Net of 
Fall Score, Teacher 
Experience, Gender, 

& Certification Status 

Magnitude of 
Effect {d) in Standard 
Deviation Units'* 

Technology Proficiency 

Yes 

0.16 (small) 

Professional Productivity 

Yes 

0.37 (moderate) 

Students’ Technology Use 

Yes 

0.70 (large) 

Integration 

Technology Integration 

Yes 

0.73 (large) 

Learner-Centered Instruction 

Yes 

0.30 (moderate) 

Resistance to Integration 

No 

-0.16 (small) 

Collaboration 

Yes 

0.41 (moderate) 


Source: Online teacher surveys conducted in fall 2004 and spring 2005. 

Note. Effect size is Cohen’s d. The interpretation is that anything greater than 0.5 is large, 0.5-0. 3 is 
moderate, 0.3-0. 1 is small, and anything smaller than 0.1 is trivial. 


Technology Proficiency 

Interestingly, in spring 2005, there was little difference between teachers in immersed and control 
schools on items measuring technology operations (e.g., send email to coworkers, parents, or peers; 
search for and find a Weh site; find primary sources of information on the Internet). Teachers also 
reported the highest levels of proficiency in these types of skills (5 to 6 on the 7-point scale). The 
strongest divergence between groups was treatment teachers’ higher self-reported ratings on items 
related to classroom instruction. For example, treatment teachers reported an enhanced ability to: 

• use the computer to create a slideshow presentation (M = 5.6, -1-0.3 points), 

• teach my students about copyright issues as the relate to the Internet (M = 4.9, -1-0.3 points), 

• create a lesson plan or unit that incorporates subject matter software as an integral part (M = 
4.8, -1-0.3 points), 

• use technology to collaborate with other colleagues who are distant from my classroom (M = 
4.8, -1-0.3 points), and 

• describe five software programs that I would select and use in my teaching (M = 4.4, -1-0.3 
points). 
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Table 5.4. Immersion (Fixed) Effect Analyses of Teacher-Level Variables 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Base 

666 

0.211 

22.16*** 


Fall score (pooled) 

0.753 

0.010 

38.71*** 


Experience (pooled) 

-0.007 

0.003 

-2.13* 


Certification status 

0.016 

0.206 

0.08 


Gender 

0.124 

0.055 

2.25* 


Immersion dummy 

0.222 

0.071 

3.12** 

Professional Productivity 


Base 

3.018 

0.117 

25.90*** 


Fall score (pooled) 

0.692 

0.032 

21.90*** 


Experience (pooled) 

-0.003 

0.002 

-1.64 


Certification status 

-0.002 

0.103 

-0.02 


Gender 

0.087 

0.039 

2.24* 


Immersion dummy 

0.273 

0.043 

6.35*** 

Students’ Technology Use 


Base 

1.708 

0.154 

11.12*** 


Fall score (pooled) 

0.653 

0.026 

25.22*** 


Experience (pooled) 

-0.006 

0.002 

-2.95** 


Certification status 

0.224 

0.143 

1.57 


Gender 

0.046 

0.042 

1.09 


Immersion dummy 

0.573 

0.063 

9.09*** 

Technology Integration 


Base 

2.424 

0.185 

13.08*** 


Fall score (pooled) 

0.648 

0.031 

20.68*** 


Experience (pooled) 

-0.012 

0.003 

-3.56** 


Certification status 

0.592 

0.152 



Gender 

0.291 

0.071 

4 08*** 


Immersion dummy 

1.115 

0.134 

8.35*** 

Learner-Centered Instruction 


Base 

3.010 

0.270 

11.15*** 


Fall score (pooled) 

0.527 

0.027 

19 25*** 


Experience (pooled) 

-0.007 

0.004 

-1.94 


Certification status 

0.602 

0.246 

2.45* 


Gender 

0.261 

0.088 

2.96** 


Immersion dummy 

0.407 

0.083 

4 

Resistance to Integration 


Base 

2.407 

0.269 

8.95*** 


Fall score 

0.497 

0.041 

12.07*** 


Experience (pooled) 

0.004 

0.004 

0.92 


Certification status 

0.145 

0.258 

0.56 


Gender 

-0.249 

0.085 

-2.95** 

Collaboration 


Base 

2.042 

0.161 

12.66*** 


Fall score (pooled) 

0.595 

0.025 

23.93*** 


Experience (pooled) 

-0.002 

0.002 

-1.07 


Certification status 

0.314 

0.158 

1.98* 


Gender 

0.063 

0.043 

1.46 


Immersion dummy 

0.321 

0.056 

5.69*** 


*p < .05; **p < .01; ***p < .001. 
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Professional Productivity 

This scale measured the frequency of teachers’ use of technology for administrative and classroom 
management, communication, information gathering, assessment, and instructional purposes. In 
spring, there were substantial differences between teacher groups in some key areas. Treatment 
teachers used technology far more often than control teachers to: 

• communicate with students (M = 3.3 on the 5-point scale, -1-0.8 points), 

• post homework, class requirements, or project information on a website 
(M = 2.9, -tO.7 points), 

• administer online assessments, (M = 2.4, -1-0.6 points), 

• access model lesson plans integrating technology (M = 3.4, -1-0.3 points), and 

• deliver information using presentation software (M = 2.9, -1-0.3 points). 

Students’ Technology Use 

On this scale, teachers reported how often students in their typical class used technology in various 
ways during class time. The effect size for this scale was large because treatment teachers reported 
more frequent technology use on all items. The largest differentials indicated that teachers in 
immersed schools more often than control teachers had students use technology to: 

• express themselves in writing (M = 3.4 on the 5-point scale, -1-0.9), 

• learn and practice skills (M = 3.4, -1-0.8 points), 

• conduct Internet research on an assigned topic {M - 3.3, -1-0.8 points), 

• conduct multimedia research (reference CDs, online encyclopedias), (M = 2.8, -1-0.8 points), 

• create and make presentations (M = 2.7, -1-0.6 points), 

• visually represent or investigate concepts (M = 2.5, -1-0.6 points), and 

• produce multimedia reports/projects (M = 2.4, -1-0.6 points). 

Integration 

Measurement of integration encompassed three latent variables: Technology Integration, Learner- 
Centered Instruction, and Resistance to Integration. Teachers rated items on a 7-point scale from 1 {not 
at all true of me now) to 7 {very true of me now). 

Technology Integration. This scale included items gauging teachers’ actions supporting curricular 
and instructional infusion of technology. Strong differences emerged between comparison groups on 
all items. Teachers in immersed schools were far more likely than control teachers to report: 

• I allocate time for students to practice their computer skills (M = 4.7, -t-1.6 points), 

• I plan computer-related activities in my classroom that will improve my students’ basic skills 
(M = 5.0, -1-1.4 points), 

• My students discover innovative ways to use classroom computers to make a difference in 
their lives {M - 4.7, -t-1.3 points). 


52 



• Using cutting edge technology and computers, I have stretched the instructional computing 
in my classroom {M- 4.2, +1.3 points), and 

• My students authentic problem solving is supported hy a vast array of computer-based tools 
and technology (M = 3.9, +1.2 points). 

Learner-Centered Instruction. In spring, teachers in immersed schools expressed stronger 
affiliations with principles of learner-centered instruction than control teachers. Treatment teachers 
were more likely to report that students are involved in establishing individual learning goals (M = 3.7, 
+0.3 points), and their instructional approach emphasizes experiential learning, student involvement, 
and real-world experiences (M = 4.3, +0.3). Moreover, teachers in immersed classrooms indicated 
more consistent provision of alternative assessment opportunities for students (M = 4.3, +0.5 points) 
and their selection of instructional materials allowed students to use information and inquiry skills (M 
= 4.3, + 0.7 points). 

Resistance to Integration. Teachers at immersed campuses also expressed lower levels of resistance 
to technology integration. They were less likely than control teachers to indicate that using classroom 
computers is not a priority for me this school year, computers are not a necessary part of my classroom 
instruction, and the use of computers is not practical for my students. 

Overall, although teachers in immersed schools expressed stronger support than control teachers for 
integration variables, average ratings, which typically ranged between 3 and 5 on the 7-point scale, 
indicated only moderate commitment to Technology Integration and Learner-Centered Instruction. 

Collaboration 

The collaboration scale measures teacher interactions with colleagues supporting improvements in 
instructional practices, including coaching and mentoring from internal and external sources, 
developing technology lessons collectively, and exchanging information about students and their 
learning. On each of the scale items, treatment teachers indicated more frequent collaborations than 
control teachers. Collaborations that distinguished the comparison groups indicated that treatment 
teachers more often: 

• exchanged feedback with other teachers based on student work that used technology (M = 
3.1, +0.7 points), 

• consulted with other teachers about certain students’ technology skills and use 
(M =3.1, +0.6 points), 

• had informal discussions with colleagues regarding strategies for integrating technology (M 
= 3.4, +0.5 points), 

• received coaching or mentoring from an internal source, such as another teacher or 
technology coordinator (M = 3.1, +0.4 points), and 

• received coaching or mentoring from an external (non-school) source, such as a professional 
curriculum developer (M = 2.6, +0.4 points). 
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Associations of Teacher Characteristics and Mediating Variabies 


Results for HLM analyses also revealed important associations between teacher characteristics and 
technology-related variables. 

Teacher experience (net of fall scores, certification status, and gender) was a significant positive 
predictor of Innovative Culture, but a significant negative predictor of Technology Proficiency, 
Student’s Use of Technology, and Technology Integration. 

Female teachers (net of fall scores, certification status, and experience) had higher spring 2005 scores 
than male teachers on: Classroom Integration, Collegial, Innovative Culture, Parent and Community 
Support, Technology Proficiency, Professional Productivity, Technology Integration, and Learner- 
Centered Instruction. Net of the predictors, females had lower Resistance to Integration scores than 
males. 

Technology certified teachers (net of fall scores, gender, and experience) had higher spring 2005 
scores than non-certified teachers on: Leadership and System Support, Technical Support, 
Collaboration, Technology Integration, and Learner-Centered Instruction. 

Additionally, ancillary statistics in Tables 5.3 and 5.4 show that fall scores, gender, experience, and 
certification status reduced teacher-level variance in spring 2005 scores by anywhere from 20.3% 
(Parent, Community Support) to 69.2% (Technology Proficiency). Moreover, the immersion variable, 
net of the teacher-level predictors, reduced between-school variance in spring 2005 scores by 
anywhere from 16.8% (Technical Support) to 77.2% (Student’s Use of Technology). 


Table 5.5. Ancillary Statistics for HLM Analyses of School- and Teacher-Level 
Variables 


School-Level Scale 

Intraclass 
Correlation or 
Proportion of 
Outcome Variation 
Between Schools'* 

Percentage of 
Variance Explained 
by the Teacher- 
Level Model'’ 

Percentage of 
Variance Explained 
by the School-Level 
ModeP 

School-Level Variables 

Leadership and System Support 

0.130 

0.466 

0.273 

Classroom Integration 

0.141 

0.341 

0.683 

Technical Support 

0.244 

0.260 

0.168 

Innovative Culture 

0.084 

0.323 

0.549 

Parent, Community Support 

0.121 

0.203 

0.839 

Teacher-Level Variables 

Technology Proficiency 

0.028 

0.692 

0.306 

Professional Productivity 

0.093 

0.496 

0.684 

Students’ Technology Use 

0.172 

0.399 

0.772 

Collaboration 

0.117 

0.332 

0.632 

Technology Integration 

0.186 

0.473 

0.728 

Learner-Centered Instruction 

0.039 

0.311 

0.756 

Resistance to Integration 

0.061 

0.288 

0.216 


“ The intraclass correlation measures the degree of dependence in the spring 2005 scale scores among the 
teachers sharing a school. A value of 0 would indicate no need for a multilevel analysis. 

This is a measure of the proportion reduction in within-school variance computed by comparing the 
within-school variance from a null model (no predictors) with the within-school variance from the 
teacher-level model (fall 2004 scale score, teacher experience, teacher certification status, and teacher 
gender) described above. 

‘^This is a measure of the proportion reduction in between-school variance computed by comparing the 
between-school variance from the level 1 or teacher-level model with the between-school variance from 
the school-level model (immersion indicator variable as the predictor) described above. 
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Effects of Immersion on Classroom Practice 


Researchers conducted classroom observations in fall 2004 and again in spring 2005 in a sample of 
classrooms for sixth-grade teachers of core subjects (reading/English language arts, mathematics, 
science, and social studies). Classroom observations were conducted by single observers (about 75% 
of classrooms) and pairs of observers (about 25% of classrooms). Paired observations permitted the 
calculation of inter-observer agreement. Altogether, we conducted 142 observations in 22 schools in 
fall 2004 (11 schools each for treatment and control). In spring 2005, we expanded observations to 
include 235 observations in all of the 44 schools. 

During observations, data collectors recorded descriptive information about the classroom 
environment; time-interval ratings of classroom organization, teacher activities and technology use, 
student activities and technology use, student engagement, and student collaboration. Observers also 
recorded notes during the observations to capture the lesson’s content focus and objectives, teachers’ 
questioning strategies (lower and higher order), and students’ learning experiences. 

Following classroom observations, observers used time-interval ratings and descriptive notes to rate 
the Intellectual Challenge of classroom work (rating scales developed by Newmann, Secada, & 
Wehlage, 1995). One section of the Observation of Teaching and Learning (OTL) instrument included 
5-point rating scales for four dimensions of the intellectual quality of instruction: 

■ Construction of Knowledge: Higher Order Thinking. Instruction involves students in 
manipulating information about ideas by synthesizing, generalizing, explaining, 
hypothesizing, or arriving at conclusions that produce new meaning and understanding for 
them. 

■ Disciplined Inquiry: Deep Knowledge. Instruction addresses central ideas of a topic or 
discipline with enough thoroughness to explore connections and relationships and to produce 
relatively complex understandings. 

■ Disciplined Inquiry: Substantive Conversation. Students engage in extended conversational 
exchanges with the teacher or peers about subject matter in a way that builds an improved and 
shared understanding of ideas or topics. 

■ Value Beyond School: Connections to the World Beyond the Classroom. (Newmann, Secada, 
& Wehlage, 1995). 

An aggregate score across the four scales was used as an overall measure of the Intellectual Challenge 
of instruction for each teacher. In addition to training raters to enhance their observer agreement, we 
also utilized Many-Facets Rasch Analysis (Linacre, 2004) to adjust the measure of Intellectual 
Challenge for the relative severity (or leniency) of each observer. 

Table 5.6 reports the adjusted Intellectual Challenge scores for immersed and control teachers in the 
fall of 2004 and spring 2005. In fall, the control teachers’ mean adjusted score (1.75) was higher than 
the immersed teachers’ score (1.45) {t =-2.35, df- 140, p - 0.02). This difference in scores favoring 
the control teachers represented a moderate effect size (-0.39). Thus, in fall, the sample of control 
teachers engaged students in lessons involving a higher level of intellectual challenge. That is, lessons 
required a higher level of thinking, delved into topics more thoroughly, engaged students in more 
substantive conversation, and made stronger connections with students’ background experiences and 
the world beyond the classroom. However, in spring 2005, the difference between the immersed 
teachers’ mean adjusted score (1.69) and control teachers’ score (1.66) was statistically insignificant 
{t - 0.25, df - 233, p - 0.80). Thus, fall-to-spring comparisons revealed some progress by teachers in 
immersed schools toward more challenging lessons. 
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Table 5.6. Adjusted Intellectual Challenge Scores for Sixth-Grade Teachers’ Lessons In 
Immersed and Control Schools 



Immersion 

Control 



Effect 

Group 

N 

Mean 

SD 

N 

Mean 

SD 

f-value 

P 

Size 

Fall 2004 

69 

1.45 

0.85 

73 

1.75 

0.68 

-2.35 

0.02* 

-0.39 

Spring 2005 

117 

1.69 

0.85 

118 

1.66 

0.86 

0.25 

0.80 

0.03 


Note. Intellectual Challenge scores could range from 1 (lowest rating) to 5 (highest rating). *Difference is 
statistically significant. Effect size is Cohen’s d. 


Table 5.7 provides a comparison of the adjusted Intellectual Challenge scores for immersed and 
control teachers who were observed in both the fall of 2004 and again in the spring of 2005. Neither 
group’s change in adjusted scores was statistically significant {t - 1.09, df- 57, p - 0.28 for immersed 
and t = -1.18, df - 58, p - 0.24 for control). However, teachers in immersed schools showed an 
increase in the Intellectual Challenge of instruction, while the challenge of control teachers’ lessons 
decreased. Additionally, in a model predicting the spring adjusted Intellectual Challenge score from 
the fall adjusted score and a treatment variable (1 = immersion, 0 = control), the treatment effect 
favoring the immersed teachers was positive but not significant (unstandardized coefficient = 0.154, 
standardized coefficient = 0.038, t - 0.411, probability = 0.682). 


Table 5.7. Adjusted Intellectual Challenge Scores for Sixth-Grade 
Teachers In Immersed and Control Schools with Pre- and Post-Measures 



Fall 2004 

Spring 2005 



Group 

N 

Mean 

SD 

Mean 

SD 

f-value 

P 

Immersion 

58 

1.50 

0.71 

1.64 

0.83 

1.09 

0.28 

Control 

59 

1.81 

0.92 

1.65 

0.84 

-1.18 

0.24 


Note. Intellectual challenge scores could range from 1 (lowest rating) to 5 (highest rating). 
The difference between the Intellectual Challenge scores for immersion and control teachers 
in spring 2005, net of fall 2004 scores, was statistically insignificant. 
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6. Effects of Technology Immersion on Students and Learning 


In the theoretical framework for technology immersion, we hypothesized that an improved school 
environment for technology would lead to teachers who have greater technology proficiency and use 
technology more often for their own professional productivity. Moreover, in immersed schools, 
teachers have students use technology frequently in their classrooms, and technology provides a means 
to enhance the intellectual challenge and relevance of lessons. Findings reported in the previous 
section suggest that teachers in immersed middle schools, indeed, have advanced in many of these 
areas in comparison to their counterparts in control schools. Based on expected changes in teacher 
knowledge and practices in immersed schools, we also postulated that improved school and classroom 
environments for technology would lead students to greater technology proficiency and use, more 
frequent peer collahoration, enhanced personal self-direction, more challenging and relevant school 
work, and stronger engagement in school and learning. In this section, we present findings on the 
effects of technology immersion on students and their learning experiences. 


Student Mediating Variabies — HLM Anaiysis 

Data on student mediating variables come from paper-and-pencil surveys of sixth-grade students 
conducted in the fall of 2004 and again in the spring of 2005. The Student Questionnaire measured 
students’ technology proficiency, technology use, and technical problems. The survey also gauged 
students’ opportunities to work with peers in small groups and their satisfaction with school. For 
Technology Proficiency, students rated their skills in using technology applications on a 5-point scale 
ranging from 1 (/ can do this not at all or barely) to 5 (/ can do this extremely well). Students 
indicated their skill level on 22 statements aligned with the Texas Technology Applications standards. 

Students also were asked to report the frequency with which their teachers had them use specific 
technology activities (e.g., use a word processor, use a spreadsheet, create a presentation) in all of their 
English language arts, mathematics, social studies, and science classes combined. Students reported 
the frequency of technology use in 12 areas on a 5-point scale: 1 {never), 2 {rarely: a few times a 
year), 3 {sometimes: once or twice a month), 4 {often: once or twice a week), and 5 {almost daily). 
Similarly, students indicated about how often various Technical Problems (7 items) happen when they 
try to use a computer at school and the frequency of their engagement in Small-Group Work (6 items). 
Students also rated their level of School Satisfaction by indicating the extent of their agreement with 6 
statements on a 5-point scale ranging from 1 {strongly disagree) to 5 {strongly agree). Cronbach’s 
alpha coefficients for the student scale scores ranged from 0.76 to 0.94. 

In fall 2004, a total of 4,824 sixth-grade students (87%) participated in the survey. Respondents 
included 2,319 treatment students (90%) and 2,505 control students (84%). In spring 2005, 4,538 
students responded to the survey (82%), with 2,053 treatment (80%) and 2,485 control students (83%). 

Sixth graders also completed the Style of Learning Inventory (SLI) in fall and spring. The SLI 
(developed by he Metiri Group, 2004) measures Self-Directed Learning. Students rated 48 statements 
addressing their self-regulated learning on a 7-point scale ranging from 1 {completely false) to 7 
{completely true). The Cronbach’s alpha coefficient for the SLI total score is 0.88. In fall 2004, a total 
of 4,584 students (82%) completed the SLI as a baseline measure. Respondents included 2,142 
treatment students (83%) and 2,442 control students (82%). A total of 4,294 students (77%) completed 
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the SLI again in spring 2005, including 2,174 treatment students (85%) and 2,120 control students 
(71%). 

Student response rates were generally acceptable but rates tended to decline from fall to spring. Thus, 
students without pre- and post-measures were excluded from analyses. In particular, control students’ 
lower response rate compared to treatment students for the SLI in spring 2005 is troublesome (71% 
versus 85%). 

The analyses that follow contrast sixth-grade students in immersion and control schools before and 
after one school year of the implementation of the Technology Immersion Pilot. Immersion effects are 
estimated for the following scales: Technology Proficiency, Technology Use in School, Technical 
Problems, Small-Group Work, School Satisfaction, and Self-Directed Learning. The effects of 
immersion on students’ self-reported proficiencies and perceptions were analyzed using a two-level 
hierarchical linear model (HLM).^ 

Student-Level Model 

In the student-level model, spring 2005 scale scores from surveys were regressed on fall 2004 scale 
scores, economic status (0 if not disadvantaged, 1 if disadvantaged), African American status (0 if not 
African American, 1 if African American), Hispanic status (0 if not Hispanic, 1 if Hispanic) and 
gender (0 if male, 1 if female). That is, 

Yij - f^oj + /3ij(Fall 2004 scale score) + /) 2 j( Disadvantaged) + /Hs/African American) + 

/) 4 j( Hispanic) + /Us/ Female) + rij. 

When Technology Proficiency was the dependent variable, having a home computer in the fall of 2004 
(0 if family does not have a home computer, 1 if family has a home computer) was also used as a 
student-level predictor. 

With all of the 2005 student survey scales, significant variation was found across schools (although in 
cases like student Satisfaction with School the variation was not large). Variation across schools 
justifies the use of HLM. The intraclass correlations ranged from 0.083 to 0.259 (see Table 6.5). Thus, 
the school means (fioj) were specified as randomly varying. The coefficients for fall 2004 scale score 
ifiij) were specified as fixed unless the reduction in the deviance statistic (significant chi square) 
justified a random specification. The coefficients for economic status, Hispanic, African American, 
and gender were specified as fixed. 

School-Level Model 

A school-level model was developed to answer the question of whether immersed schools had higher 
scale scores than control schools, after controlling for initial scale scores, ethnicity, gender, and 
economic status. That is, 

Poj- yoo + yoi(Immersion dummy) + poj- 


HLM was used for data analysis because students are clustered within middle schools. As a result, because of selection 
processes (e.g., schools may attract similar types of students) and shared common backgrounds, students within are more 
similar to each other than are students from different schools. Consequently, measures within schools may not be 
independent, and may be more highly correlated than measures of students from different schools. (Note that when a 
clustering effect is absent, there is no need to utilize HLM.) Ignoring this clustering results in aggregation bias and mostly 
underestimated standard errors. However, hierarchical linear modeling makes no assumption about independence, and it 
estimates the degree of clustering of measures and uses this estimate in the calculation of the precision with which immersion 
effects are estimated (Raudenbush & Bryk, 2002). 
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The immersion dummy was an indicator variable with a value of 1 for an immersed school and a value 
of 0 for a control school. 


Table 6.1 provides descriptive statistics for the student- and campus-level models on the variables 
measured through the student survey (e.g., Technology Proficiency). Table 6.2 provides statistics for 
Self-Directed Learning, as measured by the SLI. 


Table 6.1. Descriptive Statistics for Student Mediating Variables 
(Survey) 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics 

Technology Proficiency (2004)“ 

4,684 

2.83 

0.91 

Technology Proficiency (2005) “ 

4,015 

3.17 

0.86 

Technology Use in School (2004) “ 

4,367 

2.05 

0.80 

Technology Use in School (2005) “ 

3,715 

2.44 

0.84 

Technical Problems (2004) “ 

4,501 

2.21 

0.93 

Technical Problems (2005) “ 

3,853 

2.45 

0.91 

Small-Group Work (2004) “ 

4,400 

2.78 

0.91 

Small-Group Work (2005) “ 

3,795 

2.85 

0.88 

School Satisfaction (2004) “ 

4,499 

3.73 

0.75 

School Satisfaction (2005) “ 

3,830 

3.72 

0.77 

Disadvantaged (1 = yes, 0 = no) 

4,782 

0.75 

0.43 

Home Computer (1 = yes, 0 = no) 

4,796 

0.68 

0.47 

Hispanic (1 = yes, 0 = no) 

4,824 

0.67 

0.47 

African American (1 = yes, 0 = no) 

4,824 

0.10 

0.30 

Female ( 1 = yes, 0 = no) 

4,824 

0.49 

0.50 

Campus-Level Descriptive Statistics 

Immersion status ( 1 = yes, 0 = no) 

44 

0.50 

0.51 


Survey items measured on a 5-point scale. 


Table 6.2. Descriptive Statistics for Self-Directed Learning (SLI) 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics 

Self-Directed Learning (2004)“ 

3,485 

4.58 

0.77 

Self-Directed Learning (2005)“ 

4,294 

4.43 

0.77 

Disadvantaged (1 = yes, 0 = no) 

4,294 

0.72 

0.45 

Hispanic (1 = yes, 0 = no) 

4,290 

0.64 

0.48 

African American (1 = yes, 0 = no) 

4,290 

0.11 

0.31 

Female (1 = yes, 0 = no) 

4,294 

0.49 

0.50 

Campus-Level Descriptive Statistics 

Immersion status ( 1 = yes, 0 = no) 

41 

0.64 

0.50 


'“Style of Learning Inventory (SLI) items measured on a 7-point scale. 
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Effects of Immersion on Mediating Variabies 


Findings from analyses are summarized in Table 6.3. Additional statistical details for the HLM models 
are reported in Tables 6.4 and 6.5. Summary results show that technology immersion had a statistically 
significant effect on students in a number of key areas and the magnitude of the effect was usually 
moderate to large. After controlling for fall 2004 scale scores, economic status, ethnicity, and gender, 
sixth-grade students in immersed schools had significantly higher scale scores than students in control 
schools on: 

• Technology Proficiency (0.40 points, 0.47 standard deviation units), 

• Technology Use in School (0.80 points, 0.96 standard deviation units), 

• Small-Group Work (0.32 points, 0.36 standard deviation units), and 

• School Satisfaction (0.10 points, 0.13 standard deviation units). 

Moreover, although students in immersed schools had access to and used technology more frequently, 
after controlling for key variables, there was no significant difference in Technical Problems reported 
by students in immersed and control schools. 


Table 6.3. Immersion Effect Analyses of Student Mediating Variables 


Scale 

Immersion Effect Net of 
Fall Score, Ethnicity, 
Gender, 

& Economic Status 

Magnitude of 
Effect (d) in Standard 
Deviation Units^ 

Technology Proficiency 

Yes 

0.47 (moderate) 

Technology Use in School 

Yes 

0.96 (large) 

Technical Problems 

No 

-0.02 (small) 

Small-Group Work 

Yes 

0.36 (moderate) 

School Satisfaction 

Yes 

0.13 (small) 

Self-Directed Learning 

No 

0.06 (small) 


Note. Effect size is Cohen’s d value. The interpretation is that an effect size greater than 0.5 is 
large, 0.5-0.3 is moderate, 0.3-0. 1 is small, and anything smaller than 0.1 is trivial. 


Student responses to individual scale items, as presented below, help to explain the positive effects of 
immersion on several mediating variables. 

Technology Proficiency 

In spring 2005, sixth graders attending immersed schools reported higher technology skill levels than 
control students on each of the items measuring the Texas Technology Application standards. For the 
most part, treatment students believed they could use technology applications either /a/r/y well or very 
well. Skill areas with the greatest mean differences favoring treatment students included the ability to: 

• send an attachment to an email (M = 3.4, -1-0.8 points); 

• use software to create a presentation (M = 4.0, -1-0.7 points); 

• open, create, modify, print, and save documents (M = 4.1, -1-0.6 points); 

• use a spreadsheet to create graphs (M = 3.4, -1-0.6 points); and 

• keep track of Web sites I have visited (M = 3.4, -1-0.4 points). 
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Technology Use in School 


Students in immersed schools also began to use technology in new ways. The greatest differences in 
technology use reported by students in spring showed that core-subject area teachers in immersed 
schools more often had students: 

• conduct Internet research on an assigned topic (M = 3.6, -i-l.O points), 

• use a word processor to write a story or report (M = 3.4, -t- 1.0 points), 

• create a presentation and present information (M = 3.2, -1-0.9 points), 

• communicate by email about topics you are studying (M = 2.7, -1-0.8 points), 

• create a database of information for a class project (M = 2.8, -1-0.7 points), and 

• use technology to complete a test or quiz (M = 2.7, -1-0.6 points). 

Even though technology use increased, frequency ratings showed that students in immersed schools 
typically participated in these technology activities only sometimes, e.g., once or twice a month. 

Technical Problems 

Problems with technology appear to be only an occasional nuisance for sixth-grade students (e.g., once 
or twice a month). Still, individual item scores indicated that control students more often had problems 
with broken or slow computers and having to share a computer to complete assignments. In contrast, 
students in immersed schools more frequently noted problems with websites blocked by a filter. 

Small-Group Work 

Students in immersed schools reported more frequent opportunities to work with their classmates in 
small-group activities than control students (e.g., tutor or coach each other, brainstorm solutions to 
problems, and discuss assignments). The differences between groups were greatest in two areas. 
Treatment students more often worked with classmates in small groups to “produce a report or 
project” (M = 3.3, -1-0.6 points) or to “make a presentation for the rest of the class” (M = 3.0, -1-0.5 
points). These activities typically occurred once or twice a month. 

School Satisfaction 

In spring, students in immersed schools expressed slightly higher levels of satisfaction in all of the six 
areas measured. The largest disparity between the comparison groups centered on two aspects of 
student self-motivated learning: “I am satisfied with the work that I do in my classes,” and “What I 
learn is more important than the grade I receive.” 

Self-Directed Learning 

We also hypothesized that the independent and individualized learning opportunities allowed through 
one-to-one technology access would have a positive effect on students’ self-direction. However, after 
controlling for fall 2004 scale scores, economic status, ethnicity, and gender, there was no significant 
difference in the spring 2005 scale scores of students in immersed and control schools on 
Self-Directed Learning. It is possible, however, that changes in students’ self-directed learning 
behaviors may require a longer period of time to emerge. Measurement error also may be a factor, 
especially considering the lower response rate for students in spring. 
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Table 6.4. Immersion (Fixed) Effect Analyses of Student Mediating Variables 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 

Base 

2.943 

0.051 

57.08*** 


Immersed campus 

0.404 

0.053 

7.58*** 


Fall score 

0.500 

0.021 

23.90*** 


Home computer 

0.106 

0.025 

4 


Disadvantaged 

-0.025 

0.029 

-0.86 


Hispanic 

-0.074 

0.035 

-2.14* 


African American 

0.026 

0.055 

0.46 


Female 

0.068 

0.029 

2.33* 

Technology Use in School 

Base 

2.002 

0.051 

39 21*** 


Immersed campus 

0.804 

0.071 

11.26*** 


Fall score 

0.294 

0.021 

13.73*** 


Disadvantaged 

0.079 

0.028 

2.86** 


Hispanic 

0.044 

0.044 

0.99 


African American 

0.214 

0.045 

4.76*** 


Female 

0.002 

0.025 

0.07 

Technical Problems 

Base 

2.453 

0.054 

45.18*** 


Immersed campus 

-0.019 

0.073 

-0.26 


Fall score (pooled) 

0.293 

0.022 

13.10*** 


Disadvantaged 

-0.022 

0.039 

-0.58 


Hispanic 

-0.012 

0.049 

-0.24 


African American 

0.007 

0.047 

0.15 


Female 

0.041 

0.033 

1.24 

Small-Group Work 

Base 

2.575 

0.048 

53.74*** 


Immersed campus 

0.318 

0.055 

3 74*** 


Fall score 

0.251 

0.018 

13.59*** 


Disadvantaged 

0.077 

0.038 

2.06* 


Hispanic 

0.021 

0.039 

0.53 


African American 

0.256 

0.047 

5 42*** 


Female 

0.103 

0.032 

3.21** 

School Satisfaction 

Base 

3.577 

0.041 

86.81*** 


Immersed campus 

0.101 

0.033 

3.09** 


Fall score 

0.393 

0.028 

13 79 *** 


Disadvantaged 

0.054 

0.029 

1.90 


Hispanic 

0.023 

0.031 

0.76 


African American 

0.062 

0.043 

1.44 


Female 

0.084 

0.032 

2.63** 

Self-Directed Learning 

Base 

4.410 

0.041 

107.42*** 


Immersed campus 

0.046 

0.039 

1.19 


Fall score 

0.617 

0.025 

24 77*** 


Disadvantaged 

-0.004 

0.030 

-0.13 


Hispanic 

-0.065 

0.034 

-1.93 


African American 

0.058 

0.047 

1.24 


Female 

0.070 

0.021 

3.39** 


*p < .05; **p < .01; ***p < .001. 
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Associations of Student Characteristics and Mediating Variabies 


Overall results for HLM analyses reported in Table 6.4 also revealed important associations between 
tbe cbaracteristics of sixth graders (i.e., gender, economic status, and minority status) and technology- 
related variables. Results indicate that: 

• Female students (net of fall scores, economic status, and ethnicity) had significantly higher 
spring 2005 scores than males on: Technology Proficiency, Small-Group Work, Self- 
Directed Learning, and Satisfaction with School. 

• A computer at home in fall 2004 (net of fall scores, gender, economic status, and ethnicity) 
was a significant predictor of a student’s Technology Proficiency. 

• Hispanic students (net of fall scores, gender, and economic status) had significantly lower 
spring 2005 scores than non-Hispanic students on Technology Proficiency. 

• African American students (net of fall scores, gender, and economic status) had significantly 
higher spring 2005 scores than non-African American students on Technology Use in School 
and Small-Group Work. 

These findings suggest that greater attention should be directed toward the learning needs of male 
students in the middle schools, both immersed and control. Moreover, results reinforce the important 
role of immersion in closing the equity gap between students who have and do not have computers in 
the home. Our data suggest that about a third of sixth-grade students did not have a computer in their 
home in fall 2004, so this disadvantage has diminished in immersed schools. For the ethnic groups, it 
is unclear whether self-reported information reflects actual differences in proficiencies and events or 
simply differences in students’ perceptions. Additional analyses may shed light on differential effects 
associated with ethnicity. 

Ancillary statistics presented in Table 6.5 show that fall scores, economic status, ethnicity, and gender 
reduced student-level variance in spring 2005 scores by anywhere from 6.2% (Technical Problems) to 
35.1% (Self-Directed Learning). The immersion variable, net of the student-level predictors (school- 
level model), reduced between-school variance in spring 2005 scores by anywhere from 0.0% 
(Technical Problems) to 75.9% (Technology Use in School). 


Table 6.5. Ancillary Statistics Related to the HLM Analyses of Student Variables 


Scale 

Intraclass Correlation or 
Proportion of Outcome 
Variation Between 
Schools'* 

Percentage of Variance 
Explained by the 
Student-Level ModeP 

Percentage of Variance 
Explained by the 
School-Level Modef 

Technology Proficiency 

0.092 

0.341 

0.604 

Technology Use in School 

0.259 

0.094 

0.759 

Technical Problems 

0.045 

0.062 

0.000 

Small-Group Work 

0.083 

0.093 

0.568 

School Satisfaction 

0.030 

0.158 

0.362 

Self-Directed Learning 

0.044 

0.351 

0.072 


“ The intraclass correlation measures the degree of dependence in the spring 2005 scale scores among the students sharing 
a school. A value of 0 would indicate no need for a multilevel analysis. 

This is a measure of the proportion reduction in within-school variance computed hy comparing the within-school 
variance from a null model (no predictors) with the within-school variance from the student-level model (fall 2004 scale 
score, ethnicity, economic status, and gender) described above. 

‘^This is a measure of the proportion reduction in between-school variance computed by comparing the between-school 
variance from the level 1 or student-level model with the between-school variance from the school-level model 
(immersion indicator variable as the predictor) described above. 
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student Mediating Variabies — Additionai Indicators 


Technology Use in Core-Subject Classes 

In addition to asking students to describe their overall use of technology in core subjects, we also 
gathered information specific to each content area (reading English/language arts, math, science, social 
studies). Measuring technology use in core subjects was important because of its link to measures of 
student achievement. To gauge the frequency of technology use, students indicated how often they 
used technology in each of their core classes (one item for each subject) on a 5-point scale: (never, 
rarely: a few times a year, sometimes: once or twice a month, often: once or twice a week, and almost 
daily). We used paired samples t-tests, as opposed to non-parametric tests which are recommended for 
analyzing ordinal data with a non-normal distribution, because the t-test is “robust against violations 
of the normality assumption” (Rasch & Guiard, 2004).* 

Results in Table 6.6 show that students in immersed schools reported significant increases in their 
technology use in all of their core-subject classes. (Note that similar conclusions would have been 
reached by using a non-parametric test.)® Effect sizes (eta squared, an alternative to Cohen’s d) for 
these increases were somewhat larger, in the moderate range, for science (0.38) and social studies 
classes (0.40) compared to other subject areas. Although students in immersed schools used 
technology most often in reading/English language arts classes (M = 2.95, nearly once or twice a 
week), the effect was small because students had used technology more often in these classes in the 
fall. On average, students used technology about once or twice a week in science (M- 2.87) and 
slightly less often in social studies (M = 2.64). Students reported using technology least often in math 
classes (M = 2.39). 


Table 6.6. Sixth Graders’ Technology Use Frequency in Core-Subject Classes 


Class 

Eall 2004 
Mean 

Spring 2005 
Mean 

t-value 

P 

Effect Size 

Immersion 

Reading/ELA 

1.92 

2.95 

26.71 

0.000 

0.279 

Math 

1.25 

2.39 

28.03 

0.000 

0.299 

Science 

1.43 

2.87 

33.94 

0.000 

0.384 

Social Studies 

1.23 

2.64 

34.91 

0.000 

0.397 

Control 

Reading/ELA 

1.89 

1.90 

0.40 

0.688 

0.000 

Math 

1.25 

1.35 

2.69 

0.007 

0.003 

Science 

1.26 

1.32 

1.60 

0.110 

0.001 

Social Studies 

1.35 

1.38 

0.87 

0.385 

0.000 


Notes. Students responded to a single item for each subject on a 5-point scale: 0 (never), 1 (rarely: a few 
times a year), 2 (sometimes: once or twice a month), 3 (often: once or twice a week), and 4 (almost 
daily). Effect size (eta squared) was calculated as dl(d -l- N - 1). Numbers ranged from 1,844 to 1,850 
for immersed students and from 2,086 to 2,104 for control students. 


* Items were ordinal measures with non-normal response distributions (significant Kolmogorov- Smirnov test). However, 
because the t test is “robust against violations of the normality assumption” (Rasch & Guiard, 2004), we used paired samples 
t tests as opposed to non-parametric tests (i.e., Wilcoxon signed ranks test) to analyze these data. 

g 

Results for Wilcoxon signed ranks tests for immersed students were significant for each core-content area: reading/ELA z = 
-22.11, p = 0.000; math z = -23.20, p = 0.000; science z = -26.61, p = 0.000; and social studies z = -27.05, p = 0.000. Results 
for Wilcoxon signed ranks tests for control students were significant for math, z = -2.50, p = 0.012; but not for reading/ELA z 
= -0.30, p = 0.764; science z = -1.62, p = 0.106; and social studies z = -0.80, p = 0.426. 
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For control students, there was a significant reported increase in technology use only in math. 
However, the effect size for this increase was almost negligible (0.003). In spring, control students 
reported that they used technology, on average, slightly more than a few times a year in math, science, 
and social studies and about once or twice a month in reading/English language arts. 

Student Engagement 

The measurement of student engagement stems from our review of the literature, with engagement in 
school and learning assessed in three ways. First, we hypothesized that students in schools immersed 
in technology would express higher levels of satisfaction with their schools and the kinds learning 
opportunities provided. Secondly, we anticipated that increased technology access and use would lead 
to improved student conduct and fewer discipline problems. Third, we posited there would be a 
positive association between immersion and improved school attendance. Findings on student 
engagement presented below suggest that students in immersed schools exhibited stronger school 
engagement on at least some indicators. 

School Satisfaction 

Students’ satisfaction with school provided one measure of engagement, and as reported earlier in the 
HLM analyses, students in immersed schools expressed significantly higher satisfaction with school 
than students in control schools, although the effect size was small (d - 0.13). 

Student Discipline and Behavior 

As an additional measure of engagement, we collected student-level data from schools in spring 2005 
on disciplinary actions that occurred during the 2004-05 school year. Disciplinary indicators included 
three variables: 

• The number of student office referrals (i.e., number of times a student was referred to the 
office for disciplinary purposes), 

• Student suspension from school (i.e., either an in-school suspension or an out-of-school 
suspension), and 

• Student placement in an alternative education setting [e.g.. Alternative Education Program 
(AEP), Disciplinary Alternative Education Program (DAEP), or Juvenile Justice Alternative 
Education Program (JJAEP)]. 

Eindings from analyses for sixth-grade students reported below suggest that students in immersed 
schools had fewer behavioral and disciplinary problems than their counterparts in control schools. 

Sent to the office during the school year. To determine whether sixth graders in immersed schools 
were sent to the office less frequently than students in control schools we conducted t-tests of 
differences between the mean numbers of office referrals. Results of the t-test were significant, t - 
6.07, p < 0.001. Specifically, there were 2,885 sixth graders in immersed schools who were sent to the 
office an average of 0.85 times, and 3,214 sixth-grade students in the control schools who were sent to 
the office an average of 1.40 times (see Eigure 6.1). The effect size for the mean difference (0.55) was 
small ((7 = 0.16). 


Analyses revealed that the distribution for the number times sixth-grade students were sent to the office throughout the 
2004-05 school year was non-normal and negatively skewed. However, because of the robustness of the t-tesl to violations of 
the normality assumption we used the parametric statistic (see Rasch & Guiard, 2004). Note that similar conclusions were 
reached with the non-parametric Mann- Whitney U test (z = -9.60, p < 0.001), with the mean rank for the immersed students, 
2,875.16 being significantly less than the mean rank for control students, 3,206.94. 
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Number of Times Sent to the Office During the Schooi Year 



Figure 6.1. Distributions of discipiinary office referrais for grade 6 students in immersion 
and controi schoois. 

We also conducted a two-way contingency table analysis to determine whether immersed (treatment) 
schools had a lower proportion of sixth-grade students who were sent to the office at least once during 
the school year. The two variables were “sent to the office at least once during the school year” (yes or 
no) and group status (treatment or control). Group status and whether or not a sixth-grade student was 
sent to the office at least once were found to be significantly related, chi-square (df = 1, N = 6,099, 
chi-square = 86.57, p < 0.001, phi = 0.12). Sixth-grade students in immersed school were sent to the 
office at a lower rate than students in control schools, although the effect was small (phi of 0.12). The 
proportions of sixth-grade students who were sent to the office at least once during the school year 
were 0.193 (19.3% or 577 out of 2,885) for the treatment schools and 0.310 (31.0% or 997 out of 
3,214) for the control schools and (see Figure 6.2). 
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Immersion or Control Group 


M Sent to office □ Not sent to office 

Figure 6.2. Number of grade 6 students sent to the office at ieast 
once by comparison group. 

Suspended during the school year. We conducted an additional two-way contingency table analysis 
to determine whether sixth-grade students in immersed schools (treatment) had a lower school 
suspension rate than students in control schools. The two variables were “suspended at any time 
during the year” (yes or no) and group status (treatment or control). Results showed there was a 
significant relationship between the school attended and whether or not a sixth-grade student was 
suspended [chi-square (df = 1, N = 6,099, chi-square = 24.98, p < 0.001, phi = 0.06)]. Sixth graders in 
immersed schools had a lower school suspension rate than did students in control schools, although the 
effect was small (phi of 0.06). The proportions of sixth graders who were suspended during the school 
year were 0.0052 (0.52% or 15 out of 2,885) for the treatment schools and 0.0196 (1.96% or 63 out of 
3,214) for the control schools (see Figure 6.3). 



Immersion or Control Group 


□ Not suspended n Suspended 


Figure 6.3. Number of grade 6 students suspended at any time during the 
schooi year by comparison group. 
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Placed in an Alternative Education Program. Another two-way contingency table analysis was 
conducted to determine whether sixth-grade students in immersed schools were less likely to be placed 
in an Alternative Education Program (AEP) than control students. The two variables were “placement 
in an AEP” (yes or no) and group status (treatment or control). Results revealed there was no 
significant difference between immersed and control schools in the likelihood of a student being 
placed in an AEP [chi-square (df = 1, N = 6,099, chi-square = 2.88, p - 0.090)]. In fact, the proportion 
of sixth graders placed in an AEP during the school year was slightly lower for control schools 
[0.0118 (1.18%) or 38 out of 3,214] than for the treatment schools [0.0170 (1.70%) or 49 out of 
2,885]. 

In general, very few sixth graders are suspended from school or placed in AEPs. It will be interesting 
to examine statistics on school suspensions and placements in alternative programs for this cohort of 
sixth graders as they advance into seventh and eighth grade. 

Student Attendance 

Another indicator of school engagement is the attendance rate of students. In Table 6.8 and Eigure 6.4, 
we compare the end-of-year attendance rates of sixth-grade students in treatment and control schools 
from one and two years prior to project implementation and after one year of project implementation. 
Results show that the average attendance rates of students in immersion schools were approximately 
0.4 to 0.5 percentage points lower than the attendance rates of control students. This pattern was 
consistent before project implementation and continued one year after project implementation. Thus, 
there was no apparent “boost” in attendance for students who participated in the project. Statistically, 
the differences between the attendance rates for treatment and control students were significant, 
although effect sizes (d) were small. 


Table 6.8. Group Differences for Sixth-Grade Student Attendance Rates 



Immersion 

Control 

Differ- 



Effect 

Year 

Mean 

SD 

Mean 

SD 

ence 

t-value 

P 

Size 

2002-03 

96.64 

3.73 

97.01 

3.35 

-0.37 

-3.80 

0.000* 

-0.10 

2003-04 

96.65 

4.91 

97.19 

3.43 

-0.54 

-4.57 

0.000* 

-0.13 

2004-05 

96.10 

4.38 

96.45 

4.28 

-0.35 

-2.89 

0.004* 

-0.08 


Source: Individual student data from TEA. 

Notes. There were 2,843 grade 6 control students and 2,434 grade 6 immersion students having attendance data 
from 2003 through 2005. *Statistically significant difference. Effect size is Cohen’s d. The effect size is 
interpreted as follows. A value greater than 0.5 is large, 0. 5-0.3 is moderate, 0.3-0. 1 is small, and anything 
smaller than 0.1 is trivial. 
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Figure 6.4. Attendance rates of sixth-grade students in immersed and controi schoois 
from 2002-03 through 2004-05. 
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7. Effects of Technology Immersion on Student Achievement 


The ultimate goal of technology immersion is increasing middle school students’ achievement in core 
academic subjects (English language arts, mathematics, science, and social studies) as measured hy the 
Texas Assessment of Knowledge and Skills (TAKS). In our theoretical framework we identified a 
sequence of causal relationships leading to improved student performance. In the model, we theorized 
that students who are enrolled in fully immersed schools will experience school and classroom 
environments that lead them to greater personal proficiencies, more intellectually challenging work, 
and stronger engagement in their school and learning. In turn, changes in students and their learning 
experiences will contrihute to increased performance on state assessments. In the first year of the 
technology immersion project, as detailed in previous sections of this report, we have cited noteworthy 
progress in some areas (e.g., changes in teacher knowledge and practice and improvements in student 
technology proficiency and school engagement). At the same time, we have pointed to limitations in 
project implementation during the initial year that almost certainly have diminished immersion’s 
potential impact on student achievement in the initial year (e.g., the small number of days that students 
actually had their laptops). 

Moreover, it is important to note that this is a longitudinal study, and while it was expected that some 
impacts might emerge in the first year, it was also considered likely that changes in student academic 
performance would require more than one year to surface. That said, the following sections present 
TAKS results for sixth-grade students who were enrolled continuously in the 22 immersion and 22 
control schools from at least October 2004 through TAKS testing in April 2005. 

Texas Assessment of Knowledge and Skills 
Passing Standards and Scale Scores 

The TAKS is Texas’ criterion-referenced assessment that measures students’ mastery of the state’s 
content standards, the Texas Essential Knowledge and Skills (TEKS). At sixth grade, the TAKS 
assesses reading and mathematics. The TAKS provides several types of scores that will be utilized in 
this study. 

■ Met the standard. This score represents satisfactory academic achievement. Students who 
meet this standard performed at a level that was at or somewhat above the state passing 
standard. Thus, students demonstrated a sufficient understanding of the knowledge and skills 
measured at the grade level. 

■ Commended performance. This score represents high academic achievement. Students who 
meet this standard performed at a level that was considerably above the state passing standard. 
Therefore, students demonstrated a thorough understanding of the knowledge and skills 
measured at this grade level. 

■ TAKS scale score. The scale score is a statistic that provides a comparison of scores with a 
standard set at 2100 for each grade level. The scale score can be used to determine whether a 
student met the minimum standard or achieved commended performance, but it cannot be 
used to evaluate a student’s progress across grades or subject areas. A scale score is provided 
for all TAKS tests. 
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Texas has adopted a phase-in plan for implementing increasingly rigorous passing standards on the 
TAKS. In 2002-03, passing was initially set at two standard errors of measurement (SEM) helow the 
State Board of Education panel recommended passing standard. In 2004-05, passing standards were 
fully implemented. Thus, for this study, TAKS scores for 2002-03 and 2003-04 have been converted 
to reflect the 2004-05 panel recommended passing standard. 

z Scores 

In addition to the scores provided hy the TEA, researchers have also generated a standard score that 
can he used to compare student progress on TAKS across grade levels. This standardized score — or z 
score — is calculated for each student and for every testing occasion and subject. The z score is 
calculated by subtracting the statewide mean grade-level scale score from each student’s scale score 
and dividing by the statewide scale score standard deviation. The z score, which has a mean of zero 
and a standard deviation of 1.0, provides a measure of TAKS score growth across grade levels and 
testing years. A student who scores at the state average will have a score of zero. A student who scores 
above the state average will have a positive score above zero, whereas a student who scores below the 
state average will have a negative score. 


Progress Toward Meeting TAKS Standards 

Information in Table 7.1 compares the absolute performance of sixth-grade students in treatment and 
control schools across two TAKS testing years. 

Table 7.1. TAKS Passing and Commended Performance Rates for Sixth-Grade 
Students 


TAKS Test 

Group 

N 

2004 

Percent 

2005 

Percent 

2005-2004 

Difference 

Met Standard 

Reading 

Immersion 

1,969 

65.5 

72.2 

6.7 


Control 

2,316 

71.6 

80.7 

9.1 

Mathematics 

Immersion 

1,990 

67.9 

56.9 

-11.0 


Control 

2,339 

71.2 

66.2 

-5.0 

Commended Performance 

Reading 

Immersion 

1,969 

17.9 

25.5 

7.6 


Control 

2,316 

22.6 

34.2 

11.6 

Mathematics 

Immersion 

1,990 

21.6 

16.4 

-5.2 


Control 

2,339 

23.5 

20.7 

-2.8 


Source: Analysis of individual student data from TEA. 

Note. The 2004 passing rates are based on 2005 standards. Students had TAKS scores in both 2004 and 
2005. Students were in the same schools for TAKS testing in the spring of 2005 and for TEA’S enrollment 
snapshot in October 2004. 

Results, which are for students with scores for both 2004 and 2005, show that students at control 
schools had higher TAKS passing rates for the reading assessment and made slightly greater gains in 
reading than students in treatment schools. About 81% of control students met the sixth-grade passing 
standard for reading compared to 72% of students at immersed campuses. Sixth graders’ passing rates 
on the mathematics assessment for both comparison groups declined in 2005; however, they decreased 
more sharply for students at immersed schools. Moreover, students’ scores on the TAKS mathematics 
assessment are notably lower than for reading. In 2005, only 57% of treatment students and 66% of 
control students met state passing standards in mathematics. 
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Only a small percentage of students met the higher “commended performance” standard. Students 
achieved commended status in reading at a higher rate than for mathematics. Comparisons between 
student groups also showed that greater proportions of control students than treatment students 
achieved the higher commended standards in both reading and mathematics. 

Table 7.2 provides sixth graders’ scale scores for TAKS reading and mathematics. As expected, scale- 
score trends are comparable to passing rates. Students in control schools had higher scale scores for 
both reading and mathematics assessments, made greater gains in reading across years, and had 
smaller decreases in mathematics scores. For both student groups, scale scores for reading increased, 
whereas scores for mathematics declined in 2005. 

Table 7.2. TAKS Mean Scale Scores for Sixth-Grade Students 


TAKS Test 

Group 

N 

2004 

Scale Score 

2005 

Scale Score 

2005-2004 

Difference 

Reading 

Immersion 

1,969 

2,164.9 

2,215.1 

50.2 


Control 

2,316 

2,199.6 

2,263.1 

63.5 

Mathematics 

Immersion 

1,990 

2,195.2 

2,145.7 

-49.5 


Control 

2,339 

2,219.4 

2,194.5 

-24.9 


Source: Analysis of individual student data from TEA. 

Note. Students had TAKS scores in both 2004 and 2005. Students were in the same schools for TAKS 
testing in the spring of 2005 and for TEA’s enrollment snapshot in the October 2004. 


As a whole, TAKS passing rates and scale scores provide evidence of student progress toward meeting 
state standards — however, additional statistical analyses are necessary to determine the effects of 
immersion on student achievement. 


Effects of Immersion on Sixth Graders’ Achievement 

The analyses that follow contrast the achievement of sixth-grade students in immersion and control 
schools before and after one academic year of implementation of the Technology Immersion Pilot. 
Immersion effects are estimated for sixth-grade TAKS reading and mathematics z scores. The z score 
provides an appropriate measure for making comparisons across grade levels and student groups. The 
effects of immersion on students’ reading and mathematics z scores were analyzed using a two-level 
hierarchical linear model (HLM). HLM is a “value added” methodology. That is, after controlling for 
students’ initial achievement and characteristics and accounting for variance at both the student and 
school level, researchers can assess the “value added” by a treatment. 

Student-Level Model 

In the student-level model, spring 2005 reading and mathematics z scores were regressed on spring 
2004 reading and mathematics z scores, minority status (I if Hispanic or African American, 0 if not), 
economic status (I if economically disadvantaged, 0 if not), and gender (1 if female, 0 if male). That 
is, 

Yy - fioj + PijiSpring 2004 z score) + /) 2 / Minority status) + /Us/ Economic status) + 

/) 4 j( Gender) + ry. 


With both 2005 reading and mathematics scores, significant variation was found across schools. 
Specifically, 10% of reading variance and 14% of mathematics variance was between schools (see 
Table 7.5). Thus, the school means (Jiloj) were specified as randomly varying. The coefficients for 
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spring 2004 z scores ifiij) were also specified as randomly varying (significant chi-square statistics). 
The coefficients for minority status, economic status, and gender were specified as fixed. 

School-Level Model 

A school-level model was developed to answer the question of whether immersed schools had higher 
achievement scores than control schools, after controlling for initial achievement and minority status, 
economic status, and gender. That is, 

Poj - yoo + yoi(Immersion dummy) + /uoj- 


The immersion dummy was an indicator variable with a value of 1 for an immersed school and a value 
of 0 for a control school. Descriptive statistics for these analyses are reported in Table 7.3. The results 
of the HLM analyses are reported in Tables 7.4 and 7.5. Findings show that: 

• After controlling for spring 2004 reading z scores, minority status, economic status, and 
gender, there were no significant differences in the spring 2005 reading z scores of students 
in immersed schools and students in control schools. 

• After controlling for spring 2004 mathematics z scores, minority status, economic status, and 
gender, there were no significant differences in the spring 2005 mathematics z scores of 
students in immersed schools and students in control schools. 

• Net of spring 2004 reading scores, minority status, and economic status, females had higher 
spring 2005 reading scores than males. 

• Net of spring 2004 scores, minority status, and gender, economically advantaged students 
had higher spring 2005 scores than economically disadvantaged students in both reading and 
mathematics. 

• Net of spring 2004 scores, economic status, and gender, non-minority students had higher 
spring 2005 scores than minority students in both reading and mathematics. 


Table 7.3. Descriptive Statistics for Sixth-Grade Achievement 


Variable Name 

N 

Mean 

SD 

TAKS Reading (Student Level) 

Gender ( 1 = female, 0 = male) 

4,285 

0.51 

0.50 

Minority (1 = minority, 0 = not) 

4,285 

0.75 

0.43 

Eco. disadvantaged (1 = yes, 0 = no) 

4,285 

0.72 

0.45 

TAKS Reading z score (2004) 

4,285 

-0.13 

0.98 

TAKS Reading z score (2005) 

4,285 

-0.26 

0.98 

TAKS Mathematics (Student Levei) 

Gender ( 1 = female, 0 = male) 

4,329 

0.51 

0.50 

Minority (1 = minority, 0 = not) 

4,329 

0.75 

0.43 

Eco. disadvantaged (1 = yes, 0 = no) 

4,329 

0.73 

0.45 

TAKS Mathematics z score (2004) 

4,329 

-0.09 

0.95 

TAKS Mathematics z score (2005) 

4,329 

-0.25 

0.91 

Campus-Levei Descriptive Statistics 

Immersion status ( 1 = yes, 0 = no) 

44 

0.50 

0.51 
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Table 7.4. Immersion (Fixed) Effect Analyses of Sixth-Grade Achievement 


TAKS Achievement Test 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

f-value 

Reading 


Base 

-0.132 

0.036 

-3.66** 


Immersion dummy 

-0.047 

0.042 

-1.13 


Spring 2004 score 

0.699 

0.019 

37.44*** 


Gender 

0.150 

0.018 

8.30*** 


Minority 

-0.104 

0.029 

-3.59** 


Eco. disadvantaged 

-0.115 

0.035 

-3.27** 

Mathematics 


Base 

-0.084 

0.046 

-1.84 


Immersion dummy 

-0.031 

0.052 

-0.60 


Spring 2004 score 

0.695 

0.022 

31.83*** 


Gender 

0.030 

0.019 

1.58 


Minority 

-0.090 

0.021 

-4 37*** 


Eco. disadvantaged 

-0.093 

0.022 

-4.14*** 

*p < .05; **p < .01; ***p < .001. 


Table 7.5. Ancillary Statistics Related to the HLM Analyses of Sixth-Grade 
Achievement 


Achievement 

Intraclass Correlation or 
Proportion of Outcome 
Variation Between 
Schools^* 

Percentage 
of Variance 
Explained by the 
Student-Level 
ModeL 

Percentage 
of Variance 
Explained by the 
School-Level 
Model" 

Reading 

0.099 

0.543 

0.047 

Mathematics 

0.139 

0.557 

0.004 


“ The intraclass correlation measures the degree of dependence in the spring 2005 z scores among the 
students sharing a school. A value of 0 would indicate no need for a multilevel analysis. 

This is a measure of the proportion reduction in within-school variance computed by comparing the 
within-school variance from a null model (no predictors) with the within-school variance from the 
student-level model (fall 2004 z score, gender, minority status, and economic status as predictors). 

‘^This is a measure of the proportion reduction in between-school variance computed by comparing the 
between- school variance from the level 1 or student-level model with the between-school variance from 
the school-level model (immersion indicator variable as the predictor). 

In sum, results for HLM analyses revealed that although students at immersed schools had somewhat 
lower TAKS z scores in reading and mathematics, there were no statistically significant differences 
between groups. There were, however, differences between schools within the treatment and control 
groups and differences in student growth rates. These differences will be the focus of additional 
statistical analyses in future reports. 
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8. Conclusions and Implications 

The study of technology immersion (i.e., a laptop computer for every middle school student and 
teacher, wireless access throughout the campus, curricular and assessment resources, professional 
development and ongoing pedagogical support for curricular integration, and technical support for 
immersion) adds substantially to the body of research on the impacts of technology, especially 
students’ one-to-one access to computers for learning. First-year results reveal positive effects on 
schools (leadership and system support, innovative culture, classroom integration, parent and 
community support), teachers (proficiency and productivity, technology use and integration, 
collaboration), and students (technology proficiency and use, small-group work, school satisfaction, 
and behavior). Findings are generally consistent with the research literature on technology and one-to- 
one computing initiatives, and in most cases, the sizes of effects suggest that the impacts of technology 
immersion are of both statistical and practical importance. In contrast to positive effects on school, 
teacher, and student mediating variables, there were no statistically significant effects of immersion in 
the first year on either reading or mathematics achievement for sixth graders, who are members of a 
student cohort that will be followed through eighth grade. 

Overall, positive findings are compelling in light of evidence indicating that the level of 
implementation in the first year for 20 of the 22 middle schools was only partial immersion (2 on a 4- 
point implementation scale) rather than substantial (2 schools) ox full immersion (no schools). 

Reported effects reflect outcomes for middle schools with predominantly minority (69%) and 
economically disadvantaged students (71%). Schools are typically small (394 students, on average); 
however, enrollments vary widely (from 100 to 1,447 students). Although middle schools are highly 
concentrated in rural and very small Texas districts, about a third of districts and schools are in large 
cities or suburban locations across the state. 

Altogether, this research provides a large-scale, methodologically rigorous study of the impact of 
ubiquitous technology on students, teachers, and schools. The theoretical framework guiding the 
research has allowed us to build on and extend the existing knowledge base. The study’s quasi- 
experimental design, however, compels researchers to demonstrate that detected effects are not 
attributable to pre-existing differences in groups. Considering the fact that no large, statistically 
significant treatment-control group differences emerged from a comprehensive set of baseline 
measures, we believe comparison groups were sufficiently well matched to infer effects. Still, because 
treatment campuses enrolled slightly larger proportions of disadvantaged students, we have used 
statistical controls in analyses to adjust for potentially confounding variables. On the whole, 
researchers are confident that reported effects can be attributed to the treatment. 

A primary limitation of the study is generalizability. Compared to Texas middle-school students as a 
whole, students in the sample schools are substantially more Hispanic and less white and African 
American. Middle schools are also smaller than the statewide average (413 students, on average, vs. 
609) and are concentrated in smaller districts (18,737 students, on average, vs. 27,000). Additionally, 
for many variables, the study relies on self-reported data from surveys of teachers and students — thus, 
some findings on changes in proficiencies and practices reflect respondents’ perceptions. Nonetheless, 
the triangulation of data from multiple sources (surveys, classroom observations, field work, and state 
databases) lends greater credence to findings. Given the contextual conditions and study limitations, 
key findings from the first year are described below. 
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First-Year Implementation 


In the first year, almost all middle schools achieved only partial immersion. Middle schools 
struggled in the initial year to accommodate the complex demands of technology immersion within the 
existing school environment. As might he expected, no campus reached full immersion. However, 
some middle schools made greater strides toward immersion than others across four domains (Robust 
Access to Technology, Technical and Pedagogical Support, Professional Development, and Resource 
Utilization j. The two middle schools that achieved substantial immersion had stronger district and 
campus leadership for the project and invested greater time and resources in professional development. 

In general, first-year implementation was affected hy a number of school and contextual factors. First, 
time for planning was insufficient due to grant-related logistical procedures. Furthermore, many 
middle schools, which were housed in older buildings, encountered problems with outdated 
infrastructures and technical problems with wireless networks and Internet connectivity. Districts and 
campuses also had to grapple with myriad policies and practices related to laptop access and use. 

The greatest barriers to the implementation of technology immersion, however, involved people. 
Teachers were at different stages of readiness for immersion and their receptivity varied. Varying 
abilities and attitudes, coupled with teachers’ perceived pressures to improve students’ scores on the 
Texas Assessment of Knowledge and Skills (TAKS), made many teachers reluctant to try new and 
untested instructional methods and materials in the first year. Additionally, leadership at both district 
and campus levels emerged as a critical factor driving or limiting progress. Leadership for schools 
varied substantially due to administrative turnover or absence, the degree of commitment to the 
project, and the presence of leadership teams. Despite considerable challenges, a number of positive 
outcomes surfaced in the first year. 

Effects of Immersion on Schools 

Technology immersion positively affects the school culture, including factors such as innovation, 
collahoration, leadership, parent and community support, and students’ school satisfaction. 

Similar to other studies, our results show that the infusion of technology can be a catalyst for school 
change (e.g.. Baker et ah, 1994; Dwyer, 1994). Technology immersion had a statistically significant 
effect on teachers’ perceptions of four school-level factors. Since immersed schools received a wealth 
of technology resources over the course of the year, it was predictable to find that teachers perceived 
greater school-level availability and use of resources for Classroom Technology Integration than 
control teachers (effect size of 0.56). Teachers in immersed schools also reported stronger Leadership 
and System Support for technology (effect size of 0.20). More remarkable, however, was immersed 
teachers’ perceptions of a more Innovative Culture in their middle schools compared to control 
teachers (effect size of 0.35). In particular, teachers at immersed schools were more likely to share an 
understanding about the use of technology to enhance student learning, and they were less afraid to 
learn about and try new technologies in their classes. The infusion of technology also increased 
Collaboration among treatment teachers (effect size of 0.41). Teacher interactions at immersed schools 
significantly more often than at control supported improvements in instructional practices, including 
coaching and mentoring from internal and external sources, developing technology lessons 
collectively, and exchanging information about students and their learning, for example. 

The implementation of technology immersion also generated a great deal of excitement in schools and 
communities. Schools held laptop orientation sessions for parents, and the distribution of laptops to 
students was frequently celebrated with special ceremonies involving parents, community members, 
and local and state policymakers. All of these things likely contributed to immersed teachers’ belief 
that their school has stronger Parent and Community Support for technology (effect size of 0.49). 
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Sixth-grade students at immersed middle schools also expressed significantly higher levels of School 
Satisfaction than control students (effect size of 0.13). Treatment students were more likely to he 
satisfied with their school work, consider learning more important than the grade received, and see a 
connection between school work and their future life and work. 

Effects of Immersion on Teachers 

Teachers at immersed schools perceive themselves as more technology proficient than control 
teachers and use technology more productively to support professional practices. In a self- 
assessment of their Technology Proficiency in spring, teachers at immersed schools considered 
themselves to he significantly more technology literate than control teachers (effect size of 0.16). 
Although teachers were equally likely to he proficient in technology operations, teachers at immersed 
schools reported greater pedagogical skills in areas such as creating electronic presentations, teaching 
about copyright issues, creating lessons plans integrating technology, and using technology for 
collaboration with colleagues. Immersed teachers also began to use technology more often for 
administrative and classroom management purposes. Treatment teachers reported significantly greater 
use of technology than control teachers for Professional Productivity (effect size of 0.37) on indicators 
such as communicating with students, posting information on a website, administering an online 
assessment, and accessing model lesson plans integrating technology. 

Teachers at immersed schools have students use technology more often and they report the use 
of more innovative and learner-centered practices compared to control teachers. With increased 
availability of technology, teachers at immersed schools compared to control reported in spring that 
their Students Use Technology significantly more often in their classrooms (effect size of 0.70). For 
example, students more often express themselves in writing (using a word processor), learn and 
practice skills, and conduct Internet research on an assigned topic. Still, treatment teachers’ responses 
suggest that students may do such activities infrequently (i.e., only once or twice a month). 

Teachers at immersed schools also expressed stronger support for Technology Integration (effect size 
of 0.73). For example, treatment teachers were more likely than control to report that they allocate 
time for students to practice their computer skills, plan computer-related activities to improve 
students’ basic skills, use cutting-edge technology, and use computers to promote students’ problem 
solving and critical thinking. Immersed teachers also expressed a stronger affiliation with Learner- 
Centered Instruction (effect size of 0.30). Immersed teachers, for instance, were more likely that 
control to indicate that students establish individual learning goals, engage in experiential learning, 
and have real-world experiences. 

Although teachers at immersed schools use technology more, their lessons typically lack 
intellectual challenge. Technology immersion’s theorized impact on student achievement hinges not 
just on more frequent technology use, but also on technology’s facilitation of more rigorous and 
authentic learning (e.g., high-level thinking, concept formation, inquiry and investigation, access to 
and use of a wide range of information, exposure to places and resources beyond the classroom, and 
real-world learning) (Bransford et ah, 2003; Goldman et ah, 1999; Johnson & Cooley, 2001; Sulla, 
1999). Thus, during fall and spring observations in sixth-grade core content classrooms, researchers 
rated the Intellectual Challenge of lessons. Rating scales (developed by Newmann, Secada, and 
Wehgle, 1995) assessed the extent to which lessons and activities involved Higher Order Thinking, 
Disciplined Inquiry (Deep Knowledge and Substantive Conversation), and Value Beyond School. 

Pre- and post-results for 58 immersed and 57 control teachers revealed no statistically significant 
differences between comparison groups in spring 2005. Nevertheless, fall-to- spring comparisons 
revealed that teachers in immersed classrooms provided slightly more challenging lessons in spring. 
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whereas control teachers taught less challenging lessons. More noteworthy, however, was the low 
level of intellectual challenge in class activities for both comparison groups (about 1.6 on the 5-point 
intellectual challenge scale). In many of the observed sixth-grade classrooms, with or without laptop 
use, teachers concentrated on lower order factual knowledge and skills. Lessons frequently involved 
multiple-choice or short-answer worksheets focused on the acquisition of basic skills rather than more 
complex endeavors and higher order thinking. 

Additionally, lessons often featured brief instructional segments across a variety of learning objectives 
rather than in-depth focus on a topic or concept. Moreover, teachers rarely helped students to 
understand the relevance of their learning or make connections with their prior experiences. Findings 
from classroom observations are important because of the established link between more challenging 
and authentic pedagogy and academic achievement (Newman & Associates, 1996; Newmann, Bryk, & 
Nagoaka, 2001). If abundant access to technology fails to elevate the quality of students’ learning 
experiences, the likelihood of a positive impact on student achievement may be diminished. 

A major challenge for teachers in the first year was simultaneously learning how to use 
technology and finding time to integrate laptops and digital resources into existing practices. 

Although teachers at immersed schools, as a whole, made substantial progress in the first year, teacher 
proficiency and laptop use varied greatly by teacher, subject area, and school. Decisions about how 
and how often laptops were used for teaching and learning depended on each teacher’s readiness and 
preference. Survey results show that more experienced teachers and male teachers in middle schools 
viewed themselves as less proficient, used technology significantly less often, and expressed lower 
level of support for technology integration. 

Information from classroom observations and field work also suggest that in the initial stages of 
implementation, most teachers maintained their existing pedagogical practices. Teachers typically had 
students use laptops to do the same kinds of activities they previously had completed with paper and 
pencil, such as completing worksheets, typing vocabulary words and definitions, or reviewing for 
multiple-choice tests. This finding is consistent with research showing that teachers progress through 
developmental stages while learning to create technology-infused classroom environments. Many 
teachers at immersed campuses appeared to be at the adoption or adaptation phases, as they were 
using technology to support traditional instruction or integrating new technology into traditional 
classroom practice (Apple Computer Inc., 1995). 

Teachers at immersed campuses also received a wealth of digital resources as part of their immersion 
packages. In the first year, most of the teachers participated in professional development aimed at 
increasing their familiarity with the resources. Still, teachers seldom used the new digital products. 
Many were overwhelmed by the complexity of learning to infuse new wireless technology into the 
existing curriculum, so new resources were mostly ignored. This suggests that teachers may have 
received too many resources within a short period of time. Similarly, other researchers have found that 
projects that increase technology capacity incrementally, using one additional resource or extending a 
step beyond the previous iteration, are more successful than those that involve an intensive influx of 
new technology tools. It appears that teachers have difficulty with instructional changes that are “too 
distant” from the status quo (Zhao, Pugh, Sheldon, & Byers, 2002). 
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Effects of Immersion on Students 


Students at immersed campuses are more highly engaged in school than control students. 

Increased student engagement is one of the most frequently cited benefits in the research literature for 
one-to-one computing (e.g., MEPRI, 2003; Rockman 1998;1999; Russell et ah, n.d.). Likewise, during 
campus visits, administrators, teachers, and students at immersed campuses cited greater student 
interest and motivation for school and learning as positive effects. Other findings corroborate 
anecdotal perceptions. Surveyed sixth-graders at immersed campuses in spring expressed significantly 
higher levels of satisfaction with their middle schools than control students. Additionally, sixth graders 
at immersed schools were sent to the office for disciplinary reasons at a significantly lower rate and 
had fewer school suspensions than students at control schools. Effect sizes for school satisfaction 
(0.13) and disciplinary measures (0.16, 0.06), however, were small. Also, for another indicator of 
engagement, the school attendance rate, there was no apparent boost for immersed students (effect size 
of 0.08). 

Technology immersion positively affects sixth graders’ technology proficiency and opportunity 
to use technology. As anticipated, sixth-grade students at immersed middle schools rated their 
Technology Proficiency significantly higher than control students (effect size of 0.47) on items 
measuring the Texas Technology Applications standards. Immersed students felt more capable of 
performing tasks such as sending an attachment to an email, creating a presentation, managing 
documents, using spreadsheets for graphs, and keeping track of websites. 

Immersed students’ increased proficiency apparently stems from more frequent technology use. 

Similar to their teachers, surveyed sixth graders at immersed schools reported significantly more 
frequent Technology Use in Core Subjects than control students (effect size of 0.96). However, despite 
large and important increases, immersed students’ technology use varied across classrooms and 
content areas. Treatment students reported using technology most often in reading/English language 
arts, science, and social studies classes (nearly once or twice a week) and least often in math classes 
(about once or twice a month). 

There was no apparent effect of technology immersion on student self-direction. We theorized 
that sixth graders’ opportunities for independent and self-guided learning afforded through one-to-one 
technology would positively affect students’ personal self-direction. Students completed the Style of 
Learning Inventory as a measure of self-directed learning, including items assessing processes such as 
forethought, performance/volition control, and self-reflection. Eindings in spring showed there was no 
significant difference between the Self-Directed Learning scale scores for sixth graders in immersed 
and control schools (effect size of 0.06). Nevertheless, changes in students’ perceptions of their self- 
direction may emerge as they progress to higher grade levels and perhaps use their laptops in more and 
better ways for learning. 

Effects of Immersion on Academic Achievement 

There was no significant effect of technology immersion on sixth graders achievement in reading 
or mathematics. The ultimate goal of technology immersion is increasing middle school students’ 
achievement in core academic subjects as measured by the state assessment (TAKS). In Texas, sixth 
graders complete TAKS assessments for reading and mathematics. We found that after one academic 
year of implementation, there were no positive effects of immersion on either reading or mathematics 
scores. After controlling for prior achievement and other important student characteristics, there were 
no significant differences in the spring 2005 reading or mathematics TAKS z scores of students in 
immersed and control schools. In fact, students in immersed schools had slightly lower scores than 
comparison students. 
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Several factors help to explain the discontinuity between the many positive effects noted for schools, 
teachers, and students at immersed campuses and the absence of a positive effect on student 
achievement outcomes. First, implementation fidelity was an important factor. Limited project 
implementation almost certainly influenced student outcomes (e.g., the small number of days that 
students actually had laptops, the minimal use of digital resources). In our theoretical model, we 
hypothesized that students in fully immersed schools would experience school and classroom 
environments that would lead to changes in students, which in turn, would lead to increased 
achievement. While we found noteworthy improvements in some areas (e.g., changes in teacher 
proficiency and technology use, improvements in students’ proficiency and school engagement), there 
were no positive effects on students’ personal self-directed learning, and based on classroom 
observations, the availability of laptops did not lead to significantly greater opportunities for students 
to experience more intellectually challenging lessons or to do more challenging school work. 

Furthermore, although technology use increased in the first year and surpassed control schools, laptops 
were used infrequently for learning in core subject classes, especially mathematics. Using laptops for 
lessons once or twice a week, or once or twice a month in math classes, may be insufficient to make a 
difference in achievement. Unfortunately, students in Texas middle schools do not complete a social 
studies assessment until eighth grade or a science assessment until tenth grade, so we did not have 
academic outcome measures for those content areas. Students in the first cohort, however, will 
complete writing and social studies assessments in seventh and eighth grade, respectively. 

It is also important to remember that this is a longitudinal study, and while we expected that some 
impacts might emerge in the first year, it was also considered likely that changes in student academic 
performance would require more than one year to surface. Additionally, the findings reported here 
represent only a first step in analyzing first-year data. Additional analyses will further examine the 
relationships among school, teacher, and student mediating variables and academic achievement. We 
also intend to delve more deeply into the relationships among the fidelity of implementation, 
mediating variables, and outcomes. 
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Appendix A 

Survey Items and Scale Reliabilities 


Table A.1. Items and Reliabilities for School-Level Scales 


Scale/ltem 

Cronbach’s Alpha 
Fall Spring 

2004 2005 

Leadership and System Support 

0.91 

0.92 

The principal consults with staff before making decisions about instructional technology that affect us. 



In this school, there are clear expectations that technology will be used to enhance student learning. 



The principal in my school actively encourages teachers to pursue professional development geared 
towards curricular integration of technology. 



Our school has a well-developed technology plan that guides all technology integration efforts. 



The principal is an effective leader for instructional technology in this school. 



Overall, considering the uses of technology in my school today, 1 am confident that this use is leading to 
increased student achievement. 



The principal encourages teachers to be innovative and try new methods. 



The principal is willing to support through funding or manpower teachers’ efforts at technology 
integration. 



Administrators in this school help teachers to use technology to access, analyze, and interpret student 
performance data. 



Teachers receive adequate administrative support to integrate technology into classroom practice. 



Teachers and administrators rely on research-proven teaching and learning principles in making 
decisions about technology use. 



When our school has professional development focused on technology, the principal often participates. 



Classroom Technology Integration 

0.67 

0.68 

Students have adequate access to technology resources in my classroom (e.g., digital cameras, 
scanners, projectors). 



1 incorporate the TEKS for student technology applications into my content-area lessons. 



1 have received sufficient training to incorporate technology into my instruction. 



1 use technology to assess student performance and plan instruction. 



Technical Support 

0.71 

0.71 

Most of our school computers are kept in good working condition. 



Internet connections in my class are often too slow or not working. 



My requests for technical assistance are addressed in a timely manner. 



Materials (e.g., software, printer supplies) for classroom use of computers are readily available in my 
school. 



Problems such as computers freezing or an inability to access the Internet make it difficult for me to use 
technology. 



Innovative Cuiture 

0.78 

0.79 

Teachers in this school share an understanding about how technology will be used to enhance learning. 



Teachers in this school are continually learning and seeking new ideas. 



Teachers are not afraid to learn about new technologies and use them with their class(es). 



Teachers in this school are generally supportive of technology integration efforts. 



Parent and Community Support 

0.78 

0.79 

Parents support our school’s emphasis on technology. 



The surrounding community actively supports our instructional efforts with technology. 
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Table A.2. Items and Reliabilities for Teacher-Level Scales 


Scale/ltem 

Cronbach’s Alpha 
Fall Spring 

2004 2005 

Technology Proficiency: 1 am confident that 1 can... 

0.97 

0.97 

Send email to coworkers, parents, or peers. 



Collaborate through subscribing to a discussion list. 



Create an address book to send email to several people at once. 



Send a document as an attachment to an email message. 



Use a variety of search strategies, including key word and Boolean logic to find Web pages related to 
my subject matter interests. 



Search for and find a Web site with information about the Alamo. 



Create my own World Wide Web home page. 



Keep track of Web sites 1 have visited so that 1 can return to them later. (An example is using 
bookmarks.) 



Find primary sources of information on the Internet that 1 can use in my teaching. 



Use a spreadsheet (e.g., excel) to enter and calculate numbers. 



Use a spreadsheet to create a pie chart. 



Create a newsletter using desktop publishing techniques, including graphics and text in 3 columns. 



Perform basic software application functions such as opening an application program and creating, 
modifying, printing, and saving documents. 



Plan, create, and edit documents using word processing software (e.g.. Word). 



Use the computer to create a slideshow presentation (e.g., Powerpoint). 



Plan, create, and edit databases using database software (e.g.. Access). 



Use a database to search for and sort information and create reports. 



Use graphic organizers and/or systems thinking software (Inspiration, Stella, etc.) To teach concepts. 



Use drawing or painting software (e.g., Paint, Illustrator) to create pictures. 



Create a lesson or unit that incorporates subject matter software as an integral part. 



Use technology to collaborate with other colleagues who are distant from my classroom. 



Describe 5 software programs that 1 would select and use in my teaching. 



Write a plan with a budget to buy technology for my classroom. 



Teach my students about copyright issues as they relate to the Internet including citing sources. 



Take photos with a digital camera, save them in a digitized format, and use them in an electronic 
document. 



Scan images from a print source such as a book, save them in a digitized format, and use them in an 
electronic document. 



Create products incorporating text, audio, video, and graphics using multimedia authoring programs 
(e.g., Authorware, HyperStudio). 



Professional Productivity: As a teacher, 1... 

0.93 

0.94 

Keep administrative records (e.g., attendance). 



Manage student assessment data (e.g., electronic gradebooks). 



Use technology to analyze and interpret student data to guide instruction. 



Create electronic lesson plans. 



Communicate with students. 



Communicate with parents. 



Communicate with colleagues/other professionals. 



Create instructional materials (e.g., tests, handouts). 



Gather information from the internet to create a lesson (e.g., text, video, clipart). 



Access model lesson plans integrating technology. 



Deliver information using presentation software (e.g., Powerpoint). 



Deliver information using multimedia presentations (text, audio, video, graphics). 



Post homework, class requirements, or project information on a website. 



Administer a formative assessment using the Texas Mathematics Diagnostic System. 



Administer other online assessments. 



Use the internet at home for instructional purposes. 



Use a computer to do schoolwork at home. 
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Table A.2. Items and Reliabilities for Teacher-Level Scales (continued) 


Scale/ltem 

Cronbach’s Alpha 
Fall Spring 

2004 2005 

Students’ Technology Use: Students in my class use technology to... 

0.95 

0.98 

Express themselves in writing (e.g., word processing). 



Learn and practice skills (e.g., instructional software or educational games). 



Enter, calculate, and graph information (e.g., Excel spreadsheet). 



Create a database of information for a class project (e.g., Filemaker Pro, Access). 



Create and make presentations (e.g., Powerpoint). 



Communicate by email with peers, experts, or others on topics they are studying. 



Use online discussions to gather information for an assignment (e.g., through discussion boards or 
videoconferencing). 



Conduct internet research on an assigned topic. 



Conduct multimedia research (reference CDs, online encyclopedias). 



Enhance or express conceptual understanding through simulation/modeling software. 



Visually represent or investigate concepts (e.g., through concept mapping, graphing, reading charts). 



Produce print products (e.g., desktop publishing). 



Produce multimedia reports/projects (e.g., with video, graphics, and sound editing). 



Analyze information using tools such as graphing calculators or digital microscopes. 



Design web sites or web pages. 



Complete a test or quiz (e.g., online assessments, Texas Math Diagnostic System). 



Other (specify) 



Collaboration: As a teacher, 1... 

0.90 

0.92 

Act as a coach or mentor to other teachers or staff at my school. (May include teaching in-service 
workshop in your school.) 



Receive coaching or mentoring from an external (non-school) source such as a professional curriculum 
developer. 



Receive coaching or mentoring from an internal source, such as another teacher or technology 
coordinator. 



Have informal discussions with colleagues regarding strategies for integrating technology. 



Receive feedback from other teachers based on their observations of my teaching. 



Provide feedback to other teachers based on my observations of their teaching. 



Consult with other teachers about certain students' technology skills or use. 



Exchange feedback with other teachers based on student work that used technology. 



Work with a subject-area peer to develop a lesson plan or class activity using technology. 



Work with a colleague in a different subject area to develop a lesson plan. 



Participate in a study group with other teachers on a technology-related topic. 



Technology Integration 

0.94 

0.95 

1 alter my instructional use of the classroom computer(s) based upon the newest software applications 
and research on teaching, learning, and standards-based curriculum. 



My students discover innovative ways to use classroom computers to make a difference in their lives. 



1 allocate time for students to practice their computer skills on the classroom computer(s). 



1 integrate the most current research on teaching and learning when using the classroom computer(s). 



In my classroom, students use technology-based computer and Internet resources beyond the school 
(NASA, other government agencies, private sector) to solve authentic problems. 



My students’ authentic problem solving is supported by continuous access to a vast array of computer- 
based tools and technology. 



1 plan computer-related activities in my classroom that will improve my students’ basic skills (e.g., 
reading, writing, math computation). 



It is easy for me to design student-centered, integrated curriculum units that use the classroom 
computer(s) in a seamless fashion. 



1 seek out activities that promote increased problem-solving and critical thinking using the classroom 
computer(s). 



Using cutting edge technology and computers, 1 have stretched the instructional computing in my 
classroom. 
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Table A.2. Items and Reliabilities for Teacher-Level Scales (continued) 


Scale/ltem 

Cronbach’s Alpha 
Fall Spring 

2004 2005 

Learner-Centered Instruction 

0.75 

0.80 

Students’ authentic use of information and inquiry skiiis guides the type of instructionai materiais used in 
my ciassroom. 



My students are invoived in estabiishing individuai goais within the ciassroom curricuium. 



In addition to traditional assessments, 1 consistently provide alternative assessment opportunities that 
encourage students to “showcase” their content understanding in nontraditional ways. 



My instructional approach emphasizes experiential learning, student involvement, and students solving 
“real-world” issues. 



Resistance to Integration 

0.70 

0.72 

1 do not find computers to be a necessary part of classroom instruction. 



Using the classroom computer(s) is not a priority for me this school year. 



1 do not find the use of computers to be practical for my students. 
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Table A.3. Items and Reliabilities for Student-Level Scales 


Scale/ltem 

Cronbach’s Alpha 
Fall Spring 

2004 2005 

Technology Proficiency: How far along are you in learning to... 

0.94 

0.94 

open, create, modify, print, and save documents 



use a digital camera and/or scanner to get pictures into the computer 



send a document as an attachment to an email 



keep track of Web sites 1 have visited so that 1 can return to them later (using bookmarks, etc.) 



enter information on the computer using proper keyboarding skills 



gather information from CD-ROMS 



use online reference databases (online encyclopedias, newspapers, Library of Congress, etc.) to 
gather information 



use a search engine to find information about a topic (Alamo, etc.) on the Web 



narrow Web searches using key words and Boolean logic (such as “or,” “and,” or “not”) 



use online discussions with experts or mentors to gather information 



evaluate information found on the Web for accuracy 



use a word processor (AppleWorks, Word, etc.) to write and print a story or report 



use a spreadsheet (AppleWorks, Excel, etc.) to enter and calculate numbers 



use a spreadsheet to create graphs 



use a database (AppleWorks, Access, etc.) to enter information 



use a database to search for and sort information and create reports 



use software (Keynote, PowerPoint, etc.) to create a presentation 



use drawing or painting software (Paint, Illustrator, etc.) to create pictures 



use a video camera to make a video 



use software (HyperStudio, Authorware, etc.) to create a multimedia product 



use email to send and receive messages 



use software (FrontPage, Publisher, etc.) to create web pages 



Technology Use in School: In your English language arts, mathematics, social studies, and science 
classes, how often do your teachers have you... 

0.90 

0.92 

use a word processor (AppleWorks, Word, etc.) to write a story or report. 



use software to learn and practice skills (Riverdeep, Compass Learning, PLATO Learning, etc.). 



use a spreadsheet (Excel, etc.) to enter and calculate numbers or create graphs for an assignment. 



create a database of information (Filemaker Pro, Access, etc.) for a class project. 



create a presentation (PowerPoint, etc.) and present information to classmates or others. 



communicate by email with friends, experts, and others about topics you are studying. 



use online discussions to gather information for an assignment (discussion boards, 
videoconferencing, etc.). 



conduct Internet research on an assigned topic. 



use tools, such as graphing calculators or digital microscopes, to analyze information. 



produce print products (with desktop publishing software). 



create multimedia reports or projects (with video, graphics, and sound editing). 



use technology to complete a test or quiz. 



Other 



Technical Problems 

0.83 

0.85 

The computer is broken or slow. 



The program 1 need is not on the computer. 



The Internet connection is too slow or not working. 



A website 1 need is blocked by a filter. 



Sharing a computer makes it hard to finish assignments. 



My teacher can’t fix things when something goes wrong. 



Other (describe) 
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Table A.3. Items and Reliabilities for Student-Level Scales (continued) 


Scale/ltem 

Cronbach’s Alpha 
Fall Spring 

2004 2005 

Small-Group Work: When students work together in small groups in my classes, we... 

0.80 

0.83 

review and give advice on each other’s work. 



tutor or coach each other on difficult work. 



make a presentation for the rest of the class. 



brainstorm solutions to problems. 



discuss previous class assignments. 



produce a report or project. 



School Satisfaction 

0.77 

0.82 

1 am satisfied with the work that 1 do in my classes. 



1 understand why 1 am doing the things we do in my classes. 



The things we do in my classes will help me as an adult. 



The work we do in my classes will be useful to me in the job 1 hope to have as an adult. 



1 work hard in my classes because the work is meaningful. 



What 1 learn in my classes is more important than the grade 1 receive. 



Self-Directed Learning 

0.88 

0.89 

If I'm confused in class, 1 ask the teacher or another student for help. 



Sometimes, if 1 think an assignment is too tough, 1 purposely don't try hard. Then if 1 don't do well, 1 
don't feel bad. 



At the end of a project or assignment. I'll think about how hard 1 worked and whether 1 would do 
anything differently next time. 



It's important to me that 1 understand my schoolwork really well. 



Even when 1 think my schoolwork is boring, 1 keep working until I'm finished. 



Before 1 begin studying, 1 think about or list the things I'm going to do during my study time. 



Even when I'm supposed to learn about something boring, 1 keep working until 1 finish. 



When my teacher writes comments on assignments, 1 don't read them unless 1 have to. 



When we start a new unit, 1 like to know what we're going to be learning and how I'll know if I've 
learned it well. 



When the teacher calls on me, and 1 make a mistake in class, 1 can honestly say that 1 don't feel bad. 



When 1 do well on a big project, it's because I've worked hard. 



1 work harder than 1 need to on my schoolwork, because that's just the way 1 am. 



I'll recopy my notes or make diagrams of what we're learning to try and remember it better. 



1 don't like asking for help with my schoolwork. 



If a topic is too hard, it's really hard for me to stay motivated. 



If 1 know I'm going to do badly on a task, 1 try to avoid it, even if 1 know I'd learn a lot from it. 



There are some subjects I'm just bad at. 



A lot of times. I'll wait until the last minute to do my homework or study for a test. 



1 know 1 can make a schedule to get my work done on time and stick to it. 



When I'm doing homework, 1 rush to finish if 1 have ,a friend coming over or if a good TV show is 
about to start. 



I'll look through mistakes 1 made on earlier assignments so 1 don't make the same mistakes on new 
assignments. 



When I'm done writing a report, 1 read it over carefully and think 0 about whether I've done a good 
job. 



Even if 1 try, 1 can't make myself concentrate on schoolwork when there are more interesting things 
to do. 



When I'm reading a chapter, 1 ask myself questions to make sure 1 understand the material. 



There are some subjects 1 just can't understand, even if 1 try hard. 



When 1 get a bad grade, 1 feel dumb. 



I'll pick a tough project where 1 would learn a lot over an easy project, even if it means I'll have to 
work harder to get a good grade 



This happens to me a lot: I'll study for a test and think 1 understand everything; then 1 take the test 
and don't do very well. 



1 don't really take notes when I'm reading something for school. 
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Table A.3. Items and Reliabilities for Student-Level Scales (continued) 


Scale/ltem 

Cronbach’s Alpha 

Self-Directed Learning 



When 1 get a grade 1 don't like, I'll spend time trying to figure out what 1 could have done differently. 



When 1 do badly on a project, 1 feel okay as long as 1 did better than some of the other kids in my 
class. 



When 1 answer a question wrong in class, 1 end up wishing I'd never spoken up. 



When 1 get a bad grade, it's because 1 could have studied more or because 1 should have done 
something differently, like taking better notes. 



If I'm having trouble concentrating, 1 find a place to study where 1 won't be distracted. 



The things we're learning in my class are usually really interesting. 



If 1 have to choose. I'd rather get good grades in a class than learn a lot. 



When a big project or report is assigned, 1 make a mental or written schedule to make sure 
everything gets done on time. 



I'll usually ask someone (like my parents, friends or teacher) to give me feedback on my ideas when 
I'm working on a big assignment. 



1 know from past experience exactly what 1 have to do (like schedule a certain amount of time, or 
take notes in a particular way) if 1 want to do well on my sohoolwork. 



If an assignment isn't going to count toward my grade, 1 don't need to know how well 1 did on it. 



1 only feel bad about a low grade if 1 think 1 didn't work hard enough, or if 1 think 1 made careless 
mistakes 



When 1 read, 1 put the important ideas into my own words. 



When I'm not feeling motivated, 1 can't, make myself study. 



When 1 don't understand things in class, 1 end up thinking it's because I'm not that smart. 



When we have a reading assignment. I'll read through it one time, but 1 don't really go back through 
it to check how well 1 remember it. 



1 know 1 can do well in school if 1 try hard enough. 



1 don't ask for help, even if 1 don't understand the directions for an assignment. 



1 wouldn't do any homework if 1 didn't have to. 
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Appendix B 

Implementation Fidelity of Technoiogy Immersion 


Scoring rubrics have been developed to measure the implementation fidelity of technology immersion 
(see Table B.l). Rating scales identify four levels of immersion: minimal (1), partial (2), substantial 
(3), and full (4). The overall level of technology immersion is a composite score derived from values 
for four domains: (a) Robust Access to Technology, (b) Technical and Pedagogical Support, 

(c) Professional Development, and (d) Resource Utilization. For each domain, a number of indicators 
are used to assess the quality of implementation. Scores are derived from various data sources 
including vendor records, interviews with principals and technology specialists, focus groups with 
teachers and students, surveys of teachers and students, and grant evaluation documents. (See Table 
B.2 for a full list of data sources, items, and scales used in the measurement of implementation 
indices.) Measurement of the four domains and related elements is described below. 

Robust access to technology. Indicators for Robust Access to Technology assess the nature and 
extent of student access to laptop computers as well as the frequency of student laptop use for core- 
content learning. Four indicators — School Access, Home Access, Application use in Core Subjects, 
and Technology Frequency in Core Subjects — contribute to the domain score. 

• School Access. In an immersed school, students have access to wireless laptops the entire school 
year. In the first implementation year, however, student access to laptops varied substantially by 
school due to variations in grant award dates, time requirements for planning and purchasing 
equipment, and school policies. As a measure of School Access, we determined the number of 
days out of the 180-day school year that students actually had laptops available for use. The 
frequency distribution of available laptop days (derived from vendor and campus reports) was 
divided into quartiles to create four implementation levels: minimal (less than 90 days), partial (90 
to 120 days), substantial (121-174 days), and fully immersed (175-180 days). 

• Home Access. On a fully immersed campus, students have access to wireless laptops for learning 
both in and outside of school (24 hours a day/7days a week). Nevertheless, on some middle school 
campuses in the first year, school policies, insurance issues, parent refusals to accept liability, and 
penalties for student discipline and behavior problems limited student’ access to laptops. 
Information from interviews and focus groups revealed four levels of student Home Access: 
minimal (no access to laptops outside of school), partial (restricted laptop access outside of 
school, such as for special assignments), substantial (almost complete access outside of school, 
except for discipline or financial issues), and fully immersed (complete access to laptops outside of 
school). 

• Application Use in Core Subjects. The potential for laptops to affect achievement as measured 
by state assessments depends largely on students’ opportunities to use technology for learning core 
academic content. The student survey included 12 items measuring how often teachers have 
students use particular technology applications in their core-subject classrooms (reading/English 
language arts, mathematics, science, and social studies). For example, students rated on a 5-point 
scale how often they “use a word processor to write a story or report,” “use software to learn and 
practice skills,” and “use a spreadsheet to enter and calculate numbers or create graphs for an 
assignment.” The mean Application Use in Core Subjects score was calculated for each student 
and combined to create a school average. The 5-point scale was then divided into four levels of 
immersion for student application use: minimal (never or almost never use applications in core 
subjects, l.O-l.l), partial (rarely [a few times a year] use applications in core subjects, 1. 8-2.6), 
substantial (sometimes [once or twice a month] use applications in core subjects, 2.7-3. 5), and 
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fully immersed (use applications often [once or twice a week] or almost daily in core subjects, 3.6- 
5.0). Because students would not be expected to use each of the applications on a daily basis, the 
division of the 5-point scale into four levels reflects that reality. 

• Technology Frequency in Core Subjects. In an immersed school, technology is used 
consistently as a learning tool across the core content areas. Accordingly, four items from the 
student survey gauged on a 5 -point scale how often students used technology in each of their core 
classes (reading/English language arts, mathematics, science, and social studies classes). Mean 
student ratings were calculated and then averaged to create a mean Technology Frequency in Core 
Subjects score for each school. The 5-point scale was divided into four levels of student 
technology use: minimal (never or rarely [a few times a year] use technology in core subjects, 1.0- 
2.4), partial (sometimes [once or twice a month] use technology, 2.5-3. 3), substantial (often [once 
or twice a week] use technology, 3.4-4.2), and fully immersed (use technology almost daily, 4.3- 
5.0). 

Technical and Pedagogical Support. The provision of technical and pedagogical support for 
immersion by vendors and campus-level staff is deemed critically important, and thus, a grant 
requirement. Accordingly, we measure four indicators for support: Technical Support — Staffing, 
Technical Support — Problems, Formal Coaching and Mentoring, and Collegial Support. 

• Technical Support — Staffing. Campus-based technical support by school staff is expected to 
advance the effective use of technology for teaching and learning. Information for campus ratings 
on staffing comes from interviews with technology leaders and focus groups with teachers. From 
these sources, four levels of staffing became apparent. First, several campuses had minimal 
technical support available (little or no access to technology support staff capable of completing 
repair, assistance, or installation). Second, several campuses had access to part-time or partial 
support (limited access to capable support staff and long waits for repair, assistance, or 
installation). Third, several campuses had access to one key, full-time technology specialist, which 
amounted to substantial access (almost full access to capable support staff with moderate waits). 
Fourth, a few campuses appeared to he fully immersed (full access to capable support staff with 
minimal or no waits for repair, assistance, or installation). 

• Technical Support — Problems. On an immersed campus, a healthy infrastructure and technical 
support are expected to alleviate technical problems that might interfere with the use of technology 
in the classroom. To gauge the effectiveness of technical support, five items on the teacher survey 
assessed teachers’ perceptions of technical support (i.e., working condition of computers, speed 
and dependability of Internet connections, timely assistance, material availability, and technical 
problems). Teachers’ mean ratings on a 5-point scale of the Technical Support — Problems factor 
were averaged to provide a mean campus score. The 5-point scale was then divided into four 
levels of technical support: minimal (pervasive problems and extended waits, 0-2 A), partial 
(substantial problems and long waits, 2.5-3. 3), substantial (some problems and moderate waits, 
3.4-4.2), and fully immersed (few problems and minimal waits, 4.3-5.0). 

• Formal Coaching or Mentoring. Each middle school is expected to have a dedicated staff person 
on campus to provide ongoing pedagogical support for teachers as they learn to integrate laptops 
into teaching, learning, and the curriculum. Related, one item on the teacher survey asked each 
teacher to report how often he or she received “coaching or mentoring from an internal source, 
such as another teacher or technology coordinator” on a 5-point scale ranging from never to 
almost daily. For each campus, we calculated the percentage of core-subject teachers saying they 
received coaching or mentoring rarely or never, sometimes, or frequently (sometimes, often, or 
almost daily). Based on the distribution of scores, four levels of immersion were defined: minimal 
(50% or more of teachers rarely or never receive coaching/mentoring), partial (60% or more of 
teachers sometimes receive coaching/mentoring), substantial (60-79% of teachers frequently 
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receive mentoring/coaching), wA fully immersed (80% or more of core teachers frequently receive 
coaching/mentoring) . 

• Collegial Support. Teachers in an immersed school also support each other in their efforts to 
integrate laptops into instruction and the curriculum. Consequently, nine items on the teacher 
survey asked teachers how often they interacted with their colleagues in various ways, such as 
“informal discussions with colleagues regarding strategies for integrating technology” or 
“exchange feedback with other teachers based on student work that used technology.” Each 
teacher’s item ratings on a 5-point frequency of collaboration scale were averaged and then 
combined to provide a mean campus score. The 5-point scale was divided into four levels of 
immersion for Collegial Support: minimal (never or almost never interact with colleagues, 1.0- 
1.7), partial (rarely [a few times a year] interact, 1.8-2. 6), substantial (sometimes [once or twice a 
month] interact, 2.7-3.5), wd fully immersed (often [once or twice a week] interact with 
colleagues, 3. 6-5.0). 

Professional Development. Although the professional development models differed across immersion 
packages, each provider was expected to acquaint teachers with the resources in the immersion 
packages, support lesson development and the design of technology-enhanced learning environments, 
offer sustained learning opportunities, and provide classroom-based coaching or mentoring. Each 
school worked with the provider to develop a professional development plan for teachers and other 
staff. 

In constructing measures for the implementation of professional development models supporting 
immersion, we draw from research conducted on the effectiveness of teacher professional 
development programs (e.g.. Caret, Porter, Desimone, Birman, & Yoon, 2001; Shields, Marsh, & 
Adelman, 1998; Weiss, Montgomery, Ridgway, & Bond, 1998). Key features of quality professional 
development that have been positively associated with changes in teachers’ classroom practices 
provide a framework for examining dimensions of the vendors’ professional development models. We 
have adapted the dimensions to align with our project objectives and approaches. Measures of 
professional development elements (Contact Hours, Time Span, Collective Participation, and 
Classroom Support) come from online professional development logs submitted by districts to the 
TEA during the 2004-05 school year, documents provided by vendors, and professional development 
items included on teacher surveys. 

• Contact Hours. The duration of activities is measured as the number of technology-related 
professional development hours accrued by teachers during the school year. Contact Hours is a 
campus-level measure of the average number of hours that surveyed teachers reportedly spent in 
technology-related professional development. Teachers’ indicated their technology -related 
professional development hours on a 5-point scale (None, less than 8, 8-16 hours, 17-32 hours, 
and more than 32 hours). Teacher scores (1 to 5) were averaged to produce a campus mean level 
of contact hours. Eour levels of immersion for Contact Hours were assigned: minimal (less than 8 
hours), partial (8 to 16 hours), substantial (17 to 32 hours), wA fully immersed (32 or more 
hours). 

• Time Span. Another professional development indicator is the period of time in months over 
which the professional development activities are spread (Time Span). Using vendor and campus 
reports of the dates of professional development activities. Time Span was constructed by dividing 
the number of days that elapsed between a campus’ first and last training event by 30. Based on 
the frequency distribution for schools, the period of time in months was then divided into four 
levels of immersion: minimal (professional development events spans 1 month or less), partial 
(events span more than 1 month but less than 4), substantial (events span 4 to 8 months), anA fully 
immersed (events span 9 or more months). 
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• Collective Participation. Another indicator was the extent to which all teachers on immersed 
campuses participated in available professional development opportunities. Based on data from the 
teacher survey, we estimated Collective Participation as the percentage of teachers at each campus 
participating in 17 or more hours of technology-related professional development). Based on 
campus score distributions, we identified four levels of immersion: minimal (less than 60% of 
teachers participate in 17 or more hours of technology-related professional development), partial 
(60 to 79% of teachers participate in 17 or more hours), substantial (80 to 89% of teachers 
participate in 17 or more hours), wA fully immersed (90% or more of teachers participate in 17 or 
more hours). 

• Classroom Support. Ongoing classroom support was a required component of professional 
development models — thus, we measured the extent of Classroom Support as the percentage of 
core-subject teachers reporting that they received in-class modeling, coaching or mentoring during 
the school year. On the teacher survey, one item asked teachers to rate how often they received 
“coaching or mentoring from an external (non-school) source such as a professional curriculum 
developer.” Based on the distribution of teacher responses, we identified four levels of Classroom 
Support: minimal (50% or more of core content teachers rarely or never receive mentoring or 
coaching), partial (60% or more of teachers sometimes receive mentoring or coaching), 
substantial (60 to 79% of teachers frequently [sometimes to almost daily] receive mentoring or 
coaching, wA fully immersed (80% or more of core teachers frequently [sometimes to almost 
daily] receive mentoring or coaching from an external source). 

Resource Utilization. The technology immersion packages include a variety of instructional and 
assessment resources designed to extend, supplement, or enhance core-subject learning. Relying on 
multiple data sources, we assessed the extent to which teachers and students utilized the Productivity 
Software, Curricular Resources, and Online Assessments provided with each vendor package. 
Information on resource availability and use comes from focus group discussions with samples of 
students and teachers at each campus, observations in core-subject classrooms, and items on the 
teacher survey. 

• Productivity Software. Wireless laptops at immersed campuses are loaded with productivity 
software (i.e., either AppleWorks or Microsoft Office) for students to use as a learning tool. Data 
on the use of productivity software for instructional and learning purposes comes from four items 
on the teacher survey. Teachers reported how often students in their typical classroom “express 
themselves in writing (e.g., word processing),” “enter, calculate, and graph information (e.g.. 

Excel spreadsheet),” “create a database of information for a class project (e.g., Filemaker Pro, 
Access),” and “create and make presentations (e.g., PowerPoint, Keynote).” Teachers rated how 
often students used the productivity applications on a 5-point scale. A mean Productivity Software 
use score was calculated for each teacher and then combined to create a school average. 
Considering that certain applications were more appropriate for some core subjects than for others 
and that applications were unlikely to be used on a daily basis, the 5-point scale was divided into 
four levels of immersion: minimal (students never or almost never use productivity software, 1.0- 
IJ), partial (students rarely use productivity software, 1.8-2. 6), substantial (students sometimes 
use productivity software, 2.7-3.5), wA fully immersed (students often or almost daily use 
productivity software, 3. 6-5.0). 

• Curricular Resources. During focus group discussions at each middle school, teachers and 
students were asked to describe their use of instructional resources provided in their immersion 
packages. Additionally, researchers noted the use of instructional resources during observations in 
a sample of core-subject classrooms. From these sources, four levels of immersion for Curricular 
Resource use were defined: minimal (resources are never or rarely used for lessons or 
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assignments), /7flrf/aZ (resources are sometimes used), substantial (resources are often used), and 
fully immersed (resources are used almost daily for lessons or assignments). 

• Online Assessments. Technology immersion packages included online formative assessments in 
the core subject areas (AssessmentMaster or i-Know). Additionally, all campuses had access to the 
state’s online Texas Mathematics Diagnostic System (TMDS). Like instructional resources, data on 
the use of online assessments come from focus group discussions with teachers and students and 
observations in core-subject classrooms. From these sources, four levels of immersion for Online 
Assessments were defined: minimal (very few teachers use online assessments to diagnose student 
strengths and needs), partial (some teachers use online assessments), substantial (many teachers 
use online assessments), wA fully immersed (almost all teachers use online assessments to 
diagnose student strengths and needs). 
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Table B.1. Scoring Rubrics for Measuring the Implementation Fidelity of Technology Immersion 
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Table B.1. Scoring Rubrics for Measuring the Implementation Fidelity of Technology Immersion (continued) 
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^ Refers to teacher reports of student use of a word processor (AppleWorks, Word), spreadsheet (Excel, etc.), database (Filemaker Pro, Access, etc.), and presentation software (PowerPoint, etc.) in 
their typical classroom. 




Table B.2. Data Sources for Technology Immersion Implementation Indicators 
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Table B.2. Data Sources for Technology Immersion Implementation Indicators (continued) 
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